Forum:Cresbot

From the RuneScape Wiki, the wiki for all things RuneScape
Jump to: navigation, search
Forums: Yew Grove > Cresbot
Archive
This page or section is an archive.
Please do not edit the contents of this page.
This thread was archived on 5 April 2015 by Cqm.

As some of you may be aware, Amaurice has been winding down AmauriceBot's activity and has been looking for a new bot to take over it's remaining tasks. I have therefore created a script to allow Cresbot to take over the updating of Module:Hiscore counts. This forum seeks to formalise the changeover of that specific task.

Cresbot's source code is available here, and depends upon ceterach (the other non-standard modules can be downloaded from PyPi). At time of writing, there are still a few issues I'd like to resolve in error handling and it also needs something to automatically start the task.

I'd like to start the task midnight UTC each day. AmauriceBot ran the task every 2-3 days, but as there is a rate limit on requesting hiscore pages, which if exceeded too much can cause my IP to be blacklisted and unable to access any hiscore pages, I'd like to be able to minimise the number of requests I have to make with each run. cqm 08:38, 3 Mar 2015 (UTC) (UTC)

Discussion

Strong support name - also the functions are cool too MolMan 13:25, March 3, 2015 (UTC)

Comment - If Amaurice is winding down his bot's activity, is the Charm log bot going to need replacing too? If so, would it be possible for you to add that as well? I support allowing Cresbot to update the hiscore count. I think I wait 2-3 seconds between each request to Jamflax to avoid getting throttled. svco4bY.png3Gf5N2F.png 13:27, March 3, 2015 (UTC)

Scraping hiscore pages seems much more sensitive, I found 12 seconds gap between requests consistently avoids getting rate limited. I assume it's limited to 5 requests per minute averaged over a number of minutes, probably around 2-5 at a guess, which seems sensible and happens to be more or less the guidelines we use for editing via AWB assuming human usage.
The charm logs task is continuing for the time being, but I'm hoping to merge it into the charm drop add script as I don't see why we can't just merge the two lots of error checking into one. cqm 17:39, 3 Mar 2015 (UTC) (UTC)
Sounds good to me. svco4bY.png3Gf5N2F.png 17:40, March 3, 2015 (UTC)
I've also designed the task implementation to be extensible so it should be relatively simple to add new tasks as required, such as clearing the sandbox or simplifying the process for creating new exchange modules. cqm 17:43, 3 Mar 2015 (UTC) (UTC)
Oi, can you elaborate this on my talk page. I'd really need to know seeing how I'm pretty much 99% of charm CVU. MolMan 13:06, March 6, 2015 (UTC)

Support - Amaurice talk 10:27, March 5, 2015 (UTC)

This request for closure was denied A user has requested closure for Cresbot. Request denied. The reason given was: The discussion has ended for almost a month, and AmauriceBot is no longer updating highscores

User talk:ThePsionic.png: RS3 Inventory image of User talk:ThePsionic ThePsionic Special:Contributions/ThePsionic.png: RS3 Inventory image of Special:Contributions/ThePsionic 08:59, March 27, 2015 (UTC)

Right now I'm still working through some bugs that are preventing certain scores from being updated, one in particular is proving a little tricky to catch. I'd rather this stay open until I'm confident it can run automatically without me needing to debug and tweak it every couple of days. cqm 09:27, 27 Mar 2015 (UTC) (UTC)
Which one? MolMan 13:54, March 27, 2015 (UTC)
The particular bug that's causing me noticeable issues, it one that causes an infinite loop and eventually results in an error as it exceeds the recursion limit. As I'm using 12 seconds between requests, it takes hours for this limit to be reached and results in a huge stack trace making debugging a little more annoying that usual. cqm 21:25, 27 Mar 2015 (UTC) (UTC)
What's looping? MolMan 21:27, March 27, 2015 (UTC)
It's caused by the check here not breaking the loop (that method is recursive). cqm 21:53, 27 Mar 2015 (UTC) (UTC)

Closed - Bot appears to be stable, most major issues have either been solved or are close to being implemented. See issue tracker for more info. cqm 23:47, 5 Apr 2015 (UTC) (UTC)