Forum:Game Data Retrieval Project

From the RuneScape Wiki, the wiki for all things RuneScape
Jump to: navigation, search
Forums: Yew Grove > Game Data Retrieval Project
This page or section is an archive.
Please do not edit the contents of this page.
This thread was archived on 20 July 2021 by Legaia2Pla.



As many of you are aware, the early days of RuneScape 2 were poorly documented. After some internal discussion, the RuneScape Preservation Unit, we concluded that the best way to retrieve lost content to ask people if to check their hard drive for old caches.

Obtaining old game data would allow us to discover the following:

  • Lost music tracks
  • Unused items
  • Unused NPCs
  • Unused maps
  • Exploring anomalies such as the World map leak

The data would also help us getting a better understanding on how the game was updated over time.

I have asked Mod Ash on how Jagex would feel about this on Twitter in a private conversation: bIqeZ57.png

Our initial plan was to offer bonds as a reward, but Mod Ash is not to keen on it. Therefore, it would be best to just simply ask people without offering any sort of reward.

Our plan

Our plan is to host this page on project space in a similar fashion to One Small Wiki Favour. We would also like the following banner to shuffle with OSWF:

On our side, the RuneScape Preservation Unit will try to get content creators and expose our project to the public.

Thanks for commenting on this thread.

1wDmkih.png  Manpaint of the RPU (t)(c) 10:37, 25 June 2021 (UTC)


Oppose - I think the idea is fine, but in practice it seems like a wasted effort. First, there's not going to be a lot of people out there that can help us find old caches on their PC, imo. Maybe a small set of people would even bother to look. I just don't see any results arising from this. Secondly, the page which you've created to detail the effort goes on a little too much. It doesn't get straight to the point and tell users what they can do to help from the beginning. There's also no real way, it seems, to determine how old your cache is before you upload it to us, which means we'll probably end up having to go through a lot of work (if the project took off) to check each and every upload. We're not archaeologists, we're just humble wikians with some spare time, not time to spend all day everyday rooting through this stuff - not to mention there's only a small set of people on the wiki that have the ability/know how to read these cache files. Third, a lot of these caches are available from private server communities. I'm sure we've already looked in some of them to be able to research old things about the game. If we can't get what we need from them, given that RSPS' were very popular back in the initial years of RuneScape which your project would like to cover, then I'm not sure we'll get much better results here. There is no incentive for users to contribute towards this project, which would potentially mean booting up old hardware that they might've played RS on, so again there's not going to be a lot of people partaking in it. Finally, I'm very cautious of the legality of people sending us their cache files for the game, given that we have a close partnership with Jagex. Mod Ash is relatively small fry - just a content developer - in the whole chain of command, and it's quite possible that some lawyers there won't take too kindly to our activity. I don't think it's worth making a public project out of this. jayden 10:51, 25 June 2021 (UTC)

I don't agree with a lot of this. I think there's fairly significant value in having access to the models, maps and item/NPC/object configurations from old caches - it's already the basis for some large projects, especially on the OSRS wiki. It also seems plausible to me that in a pool of hundreds of thousands of people, a pretty decent chunk will have an old computer laying around with something maybe useful (although it remains unclear whether they can be incentivized to share). I don't see any reason that group would overlap much with the RSPS community. There are also ways we could roughly determine the age of a cache, although it would require some tooling. ʞooɔ 19:57, 25 June 2021 (UTC)
Reply - I do not really follow the logic here. On the one hand, the project is not worth it because it won't have many responses, but at the same time it will have so many responses that the workload will be too great? Regardless, checking the age of a cache takes maybe 30 seconds at most. And the fact that few resources currently exist regarding old game content is the exact point; people may have valuable old game files but simply not be aware of their potential use. I have said more about the legality argument below, but the fact that the proposed solution to legal concerns is to just work with private server communities instead is somewhat ironic. Hlwys (talk) 18:55, 1 July 2021 (UTC)

Comment - I think the overall idea here is interesting, but most of the specifics don't work. I also have concerns about whether semi-dubious legality makes it a bad idea to directly associate it with the wiki in an official capacity. I also think a single sentence on the sitenotice probably isn't enough context for people to understand if it's applicable to them, even if we wanted to make it an official thing (which I'm not sure about).

Nonetheless, I think the upside (if we can get people to check their old computers) is quite significant. I think it would work much better as something primarily in the OSRS community (rather than RS), and working with historically-inclined video creators (Colonello, willmissit, Soup, etc...) to make videos about all the cool stuff we've managed to dig up in the cache, and how anyone can help expand that further. We could make it be related to the wiki, but even then I'd probably prefer it be about "some wiki contributors" or similar, rather than "the OSRS/RS Wiki" in an official capacity.

I think the community at large would be really into that, and ~80k self-selected people watching a ~10 minute video is probably gonna give much better results than ~500k people maybe possibly looking at a sitenotice. ʞooɔ 19:57, 25 June 2021 (UTC)

Support – it’s hard to overstate how much old information and content has been lost; any efforts to retrieve or preserve this history has my support. Many projects now, such as the graphical updates pages that everybody seemed to like and are now being implemented on the RS3 Wiki, or Update history, are dependent on having older copies of the game files. These projects don’t just happen by magic – they are only possible at all because somebody in the past saved an older cache. If we want to improve or expand these projects in future, perhaps to cover periods before June 2005 for example, then finding old caches is the only way.

As of writing, RuneScape has around 120,000 players across both versions. I’m not sure how many of those know what the cache is, where it’s stored, or how to read it, but I’m assuming it’s less than 1%. Ultimately, it is a numbers game – of those 120k people, there’s bound to be a good number who have old caches but don’t even know. I was able to find a unique copy on my old computer from 2007, only because I knew what I was looking for.

Also, the argument about legality can be very easily dismissed. All this project is doing is asking people to upload a copy of the game files. For what it’s worth, RuneLite does exactly this week after week. If uploading caches really were illegal, you would think that the Wiki would have been far more careful before choosing to work with RuneLite for projects such as crowdsourcing. Hlwys (talk) 18:55, 1 July 2021 (UTC)

Oppose unless we get firm legal confirmation from Jagex - It's a potentially useful idea, but unless we get a solid confirmation (not just Mod Ash, who as far as I am aware is not a lawyer and does not represent Jagex's legal interests here), I do not support.

Aescopalus's wager

Project goes forward Project does not go forward
Legal Possible positive impact No impact
Not Legal Large potential negative impact No impact

I prefer to take the risk-adverse position. Smithing.pngAescopalus talkCrafting.png 23:20, 1 July 2021 (UTC)

Oppose running project as proposed - I'd have to agree with Cook here. The idea of obtaining these historical game files to further document historical information is intriguing and I can see the interest from editors/players. On the other hand I don't want risk any potential jeopardy to our relationship with Jagex. I can't pretend to understand any legality issues fully but Jagex' terms do include that "You may not otherwise use, decompile, reverse engineer, copy, reproduce, transmit, publicly perform, distribute, commercially exploit, adapt, translate, modify, bundle, merge, share or make available to any person, or create derivative works of the Jagex Product." and our own page on the cache warns that against sharing cache files due to the possibility of information about the player's log in details being stored there.

There is a distinction to be drawn between people using the cache to add to the wiki on an individual level in a low-key manner and running an official wiki project encouraging the community at large to share cache files with us. The second is highly likely to come to Jagex' attention and to not be viewed positively by them. I wouldn't want to show a lack of respect for the rules of the company that we have entered into a professional relationship with and I'm concerned that doing so would cause them to be less likely to trust us and want to work with us on projects in the future. Magic logs detail.pngIsobelJTalk page 11:59, 6 July 2021 (UTC)

Reply - I am glad somebody brought these terms up.

‘You may not otherwise use, decompile, reverse engineer, copy, reproduce, transmit, publicly perform, distribute, commercially exploit, adapt, translate, modify, bundle, merge, share or make available to any person, or create derivative works of the Jagex Product.’ would of course seem to rule the project out entirely. Yet as I have stated before nearly all of these are carried out by makers of third party clients, including those who worked with Jagex in the past. Furthermore, the same agreement states ‘You must not create or provide any other means by which any Jagex Product may be played by others (including, without limitation, replacement or modified client/server software or server emulators). Please note that any such activity may constitute civil wrongs and/or criminal offences, and we reserve the right to take such action as appropriate in the circumstances should we become aware that such offences are being committed.’ So not only is using third party clients against the rules, it’s also a criminal offence! Presumably Jagex are only moments away from starting legal proceedings against 70% of the OSRS playerbase.

The terms also state that users ‘to connect to our servers solely to use Jagex Products for personal and non-commercial purposes…To be clear, we consider the creation of an Account in order to access any authorised software in order to understand, decompile, reverse engineer, copy, reproduce, or transmit such authorised software as being outside the scope of “personal and non-commercial use”’ All I can say is thank goodness nobody on the Wiki has attempted to “understand, decompile, or reverse engineer” parts of the game, such as for figuring out how varbits work, otherwise that could seriously jeopardise the Wiki’s relationship with Jagex.

On a more serious note, while I understand the arguments behind the legal concerns, it honestly seems to me as if they only apply to this specific project. There are many recent projects which if taken literally would appear to be in clear breach of these rules, and (please correct me if I am wrong) I don’t remember this exceptionally over-cautious approach being taken then. Hlwys (talk) 12:40, 6 July 2021 (UTC)
Actually the part of the terms I mentioned doesn't relate to 3rd party clients. There's another part in the terms which mentions 3rd party clients specifically which the TL;DR of would be "we tolerate these but you use them at your own risk and we reserve the right to make people use the official client only". However, I think this illustrates my point about making a distinction between individual wiki editors using the cache for things vs. having "the wiki" endorse a project involving the cache and publicly advertise it to the whole of its readership. Yes, Jagex are unlikely to ever take action against individuals using 3rd party clients, but they certainly have form for holding entities to account at a more corporate level: i.e. they attempted legal against the entity responsible for creating RuneLite before because they perceived that it was against their TOS, specifically they didn't like that the game code was being tampered with. Historically this wiki has actually been very cautious in adopting the use of the cache - you can see some older yew grove discussions linked at Talk:Jagex cache where it proved to be a very divisive topic (and a large part of the difficulty seems to have been Jagex's history of being zealous in taking action against entities they perceived to be doing anything e.g. creating bots or private servers that involved using the game code and thus potentially breaking intellectual property laws).
Once again, there is a difference between the wikis approving and running a public project where players are being encouraged to do something which is against Jagex's terms and individual wiki editors choosing to assume the risk to work on cache related projects. You are probably correct in saying that technically anyone working on these projects is breaking Jagex' terms. The difference is that when individuals do this and don't make it publicly obvious it is something that Jagex either haven't noticed or don't consider enough of an issue to take action against. It is definitely worth being cautious about doing a project like the one being proposed. It would almost certainly attract (negative) attention from Jagex, with the potential implication of them taking a harder stance on any editors working with the cache as well as negatively impacting our professional relationship as I mentioned before.
If there was any attempt to talk to content creators about a project like this the distinction of it being a personal and not "wiki-approved" project would need to be maintained too in my opinion. Magic logs detail.pngIsobelJTalk page 15:21, 6 July 2021 (UTC)

Comment – That is a good point IsobelJ, and one that I believe everyone forgot (including me). The address email/username of the player is stored in a preference file attached with the cache. While it was not always the cache and that preference file can easily be detached, I do believe this kills any chance of Jagex approving this project due to the security risk. The last thing we want is people claiming they have been hacked by the wiki.

The RuneScape Preservation Unit will mobilize content creator to raise awareness about the cache and its content as Cook suggested. I suppose the thread can be closed now. 1wDmkih.png  Manpaint of the RPU (t)(c) 12:28, 6 July 2021 (UTC)

Closed - While there is not consensus regarding the opening proposal, the project does seem to be moving into a different direction. Feel free to open another thread to address further. --Legaia2Pla · ʟ · 23:46, 20 July 2021 (UTC)