Protecting technical information online requires balancing server availability with access for legitimate readers. Anubis addresses this challenge with a proof-of-work mechanism: visitors may be asked to complete a browser-based challenge before reaching a protected site. For industrial automation professionals who rely on online documentation, this approach illustrates how web scraping protection can affect access to engineering resources as well as the systems that host them.
How Anubis proof of work works
Anubis uses a scheme modeled on Hashcash, a proposed method for reducing email spam. Rather than relying only on a simple request-response check, the site asks a visitor’s browser to perform a computational task and provide proof that it has done so. The server can then use that proof as part of deciding whether to provide access.
The mechanism is designed around a difference in scale. For an individual visitor, completing the task should impose a relatively small additional burden. When requests are made at the scale of automated scraping, however, the effort accumulates across many requests. That can make large-scale collection more expensive to carry out.
This is a form of friction, not a guarantee that scraping is impossible. Its purpose is to change the economics of repeated automated access while limiting the impact on people visiting the site normally.
Why websites use a challenge as a compromise
Automated scraping can place substantial demand on a website and make its resources less available to other visitors. Anubis describes its proof-of-work challenge as a placeholder measure: it provides an interim layer of protection while more targeted ways to distinguish automated browsers from likely human visitors are developed.
One approach mentioned is fingerprinting based on browser behavior, including how a browser renders fonts. Such signals may help identify headless browsers—browsers operated without the usual visible user interface and often used for automation. The stated aim is to avoid presenting the proof-of-work page to visitors who are more likely to be legitimate.
This distinction matters operationally. A challenge that appears too often can create friction for employees, engineers, and other readers who need timely access to online material. A measure that is too easy for automated clients to bypass may provide less protection against repeated requests. The challenge therefore represents a compromise between controlling automated activity and keeping resources accessible.
Browser compatibility can affect access
The challenge depends on modern JavaScript features. Browser extensions or privacy tools that disable or restrict those features can prevent the page from completing its check. The result may be that a legitimate visitor cannot proceed, even though the site is functioning as intended from the administrator’s perspective.
For example, a user whose browser configuration blocks the JavaScript required by the challenge may repeatedly see a verification page instead of the requested content. The Anubis notice specifically identifies JShelter and similar plugins as possible causes and advises disabling them for the protected domain.
For site administrators, this highlights a practical consideration: anti-scraping controls depend not only on server configuration but also on the capabilities and settings of visitors’ browsers. Clear instructions can help users troubleshoot access problems, while compatibility should be considered when selecting or configuring a client-side challenge.
What the approach means for technical teams
For teams managing technical websites, Anubis demonstrates a layered way to raise the cost of high-volume automated access without treating every request as equally suspicious. It also shows why no single challenge should be viewed as a complete answer. Proof of work adds friction; browser fingerprinting is described as a further direction for distinguishing automated clients from likely legitimate users.
When evaluating a similar approach, consider:
- Visitor impact: How often will legitimate users encounter a challenge, and can they complete it with their typical browser settings?
- Client compatibility: Does the challenge rely on JavaScript or other browser capabilities that extensions may block?
- Protection objective: Is the goal to make mass requests more costly, rather than to claim that automated access can be eliminated?
- Support guidance: Are troubleshooting steps available for users who cannot pass the check?
These considerations are relevant wherever online resources support engineering, operations, or system integration work. A protective measure can help preserve access for a site’s broader audience, but its practical value depends on how it behaves for both automated clients and legitimate users.
Conclusion
Anubis proof of work is a browser-based way to make large-scale scraping more expensive by requiring computational effort that is relatively small at individual scale but accumulates across many requests. The approach is explicitly a compromise, not a definitive barrier, while more targeted methods such as headless browser fingerprinting are considered. For technical decision-makers, the key implication is to balance protection against automated traffic with browser compatibility, clear user guidance, and reliable access to useful information.
Access Official Technical PDF