Project Update: Adapting to Changes!
Added 2025-03-04 05:47:01 +0000 UTCHello everyone! I wanted to provide a quick project status update to keep you in the loop on what’s happening and what you can expect moving forward. As you may have noticed, many websites are now using Cloudflare as their load balancer or enhancing the obtrusiveness of their existing Cloudflare setups. Additionally, geo-blocking and frequent website updates have made scraping increasingly difficult. With that in mind, here’s what we’re currently working on:
Updating the Simulation Engine: We’re upgrading to a more recent version of the simulation engine to see if it helps address issues with Cloudflare. We hope any past problems with newer versions are resolved, but the only way to know for sure is to update and test it with user feedback.
Fixing Website Changes: We’re tackling websites that have updated or altered their behavior in ways that block content downloads. This will ensure that users can continue scraping the content they need.
Optimization: After verifying that the updated simulation engine works as expected, we’ll focus on optimizing it to ensure everything runs as smoothly and efficiently as possible. If it doesn’t perform as expected, we’ll revert to the previous version and continue refining it.
Once we’ve reviewed, updated, and fixed everything, we’ll be in a better position to move forward with feature development. Once we have a basic test build available, we'll release it to the community so that people who are adventurous can give it a try and see how it works. Thanks for your patience, and stay tuned for more updates!
As always, if you have any questions, comments, or just want to chat, please do reach out to me! While I read and reply to all messages on Patreon for Patreon specific issues, please contact me on Discord for fully unfiltered project discussions!
