Forum:SMW rework

From the RuneScape Wiki, the wiki for all things RuneScape
Jump to: navigation, search
Forums: Yew Grove > SMW rework
This page or section is an archive.
Please do not edit the contents of this page.
This thread was archived on 9 February 2019 by Gaz Lloyd.

Hey guys, I've been thinking about this for a little bit, and with the recent drop tables rework and the issues resulting from it I think now is a good time to put this into words. I didn't want to go ahead and do it since it is fairly involved and could mean some decent rewrites/additions to many core modules.

Preamble[edit source]

SMW basics[edit source]

I know that SMW is a fairly technical and seemingly complicated thing, but at the core it isn't. I've written up a summary of the basics at User:Gaz Lloyd/smw crash course.

How we use it and the problems[edit source]

The main issue is when we have versions of things on one page. This means that multiple values are set to one property - this is perfectly fine by SMW on a technical sense, but makes it a real pain to access information. Check out Special:Browse/Iban's staff - covering the broken and fixed version together means that the item value (that determines alch prices) are lumped together in 'All Value'. So to get the one you want you basically need to know what thing it is ahead of time.

There were two ways used to get around this:

  • Use multiple properties, one for each version in the infobox (see Value1 and Value2).
    • This doesn't really solve the problem. We still have 2 values and no real way to know which is which without knowing ahead of time or doing other queries to work it out.
  • Generate infobox 'dumps' as JSON and set that as a property value. Things like Item JSON, Equipment JSON, Drop JSON, Monster JSON, etc. This is how we store a lot of data.
    • This somewhat solves the issue by bundling all the information about each version together, but its cumbersome and not very nice. Primarily, if you want to use the info, you need to load it in, then decode it from the mediawiki encoding, then decode it (a different way) into a JSON object, then work out which of the JSONs is the one you care about. This also means that it requires lua to use on another wikipage, in order to decode the JSON.

If only there was a way to bundle properties together without having to create extra pages for each...

Solution![edit source]

Subobjects are the solution. Subobjects are things you can set on a page, which have their own sets of property::value pairs, but are part of an existing page, not their own page. There are two main types of subobjects: named and anonymous.

I've done some quick testing of subobjects at Noxious staff (barrows) and Noxious staff (shadow). You can check our the browse pages for them, or more easily see a list of them here. Barrows uses anonymous subobjects, shadow uses named. You can also see all properties set on the object like normal at Special:Browse/:Noxious-20staff-20(shadow)-23new (this encoding is required for the #, but you can search it in Ask just fine).

Some other (technical) things:

  • By default whenever a named subobject turns up in a list, the name is the hash in the link, which is already how we address versions (e.g. Noxious staff (shadow)#broken will automatically switch the infobox to the broken version upon load).
  • Calling the subobject parser again on the same name adds the properties to the same subobject - extremely important for versions, as infobox item, infobox bonuses sets others, and disassembly sets different groups of properties.
    • Extra importantly, this lets us set properties together in a group that we weren't able to before - we basically couldn't set a combined 'EquipItem JSON' property, as Item JSON and Equipment JSON were separated into different infoboxes. But with subobjects we can very simply bundle everything together.
  • Subobject properties do not carry over to the parent page - e.g. in the version anchor ask, or this is variant of ask, the base page (Noxious staff (shadow)) doesn't appear.
  • Subobject names can't contain a . in the first 5 characters.
  • Subobjects can have normal mediawiki categories applied to them individually - they won't appear on category pages, but this is very useful for filtering.

Proposal[edit source]

So the proposal.

Needs only backend editing (templates/modules, scripts) - very little editing of the actual pages required
  • Versioned things (items, monsters, etc) should use named subobjects, with the name being the version name.
    • Potentially review version names to make sure that . isn't used in the first 5 characters - easier to do this than have to have workarounds in the modules and switching scripts and so on
  • I am open to opinions on how to handle unversioned things:
    • For consistency we can use subobjects, using the name _main or something similar - example.
    • We could also just use the properties normally on the page, since we don't really need a subobject for them.
  • Versioned and unversioned things should also set group properties on the main page - reusing things like 'All Item ID', or new ones to do the same, so that searches can find the pages as well as subobjects.
  • DropsLine, StoreLine, and similar should use anonymous subobjects
    • We don't want or need named subobjects for these, we just want to compartmentalise the data. Rather than having to load in the full list of everything the monster drops/store sells/etc, it can just load the subobjects which contain all the information. Should make our tables much more efficient - maybe removing the (self-imposed) limit we currently have.
  • I am not sure if subobjects should contain JSON dumps of themselves - it is probably unnecessary.
Needs some editing of actual pages
  • Infobox Recipe should hook in to the method used by infobox item - notably means that versioned things with a recipe need to be told which version the recipe is for so it can put the info into the correct subobject. Otherwise it can default to putting it into the page itself/_main subobject, depending on how we handle unversioned per above.
  • Infobox Bonuses will need some of the same.
Removing old methods

At some point after the successful deployment of the new methods, we should probably look at removing some of the older methods.

  • The 'Property#' properties (e.g. 'Item ID1') should be removed, as they're entirely superseded by subobjects.
  • 'All Property' properties (e.g. 'All Item ID') are either re-purposed per above, or removed in favour of a different name.
  • 'Entity JSON' properties (e.g. 'Item JSON') are probably redundant with this and should probably be removed too.
  • We could also take this time to look at expanding SMW into things we haven't really done yet - e.g. we can set an 'Inventory image' or 'Equipped image' property, or any number of other things.
  • Use SMW in more calculators
  • Potentially rework the disassembly calculators into SMW?

Be aware that this could change during implementation due to currently unforeseen technical things.

Discussion[edit source]

Support rework Quest.png Gaz Lloyd 7:^]Events!99s 11:35, 23 January 2019 (UTC)

Support - I haven't used SMW much, especially not on this wiki, but I know how nice it can be if used well. I have a couple questions about the implementation, though:

  • Removing old methods would presumably include the Category:Tier 79 equipment and Category:Combat level 120 monsters families of categories, right?
  • I can't think of any situation where the "all X" properties as suggested are more useful than finding one or several subobjects and getting the page(s) from there. For example, using {{#ask:[[Noxious staff (shadow)#new]]|?-has subobject=|mainlabel=-}} (or likely something more general in the future) to get Noxious staff (shadow). Am I missing something?

-Hourglass (2011 Hallowe'en event) detail.png I Am Not You Talk III The Spark.png- 14:51, 23 January 2019 (UTC)

Those categories would probably stay, as they're fairly easy for readers to navigate if they're interested in looking for it, whereas the SMW would probably take more investment for a casual reader to be able to see. -- F-Lambda (talk) 20:40, 23 January 2019 (UTC)
I agree. Quest.png Gaz Lloyd 7:^]Events!99s 23:17, 23 January 2019 (UTC)
There's also a lot of lists and such that are generated via dpl using those categories. Seers headband 2 chathead.png Elessar2 (talk) 18:56, 24 January 2019 (UTC)
For the 'all X' properties, I was thinking along the lines of "give me all the item IDs associated with this page", which just adds an extra step without the all property. But I don't know if there's actually any use-cases for that at the moment, so maybe it isn't necessary. Quest.png Gaz Lloyd 7:^]Events!99s 23:17, 23 January 2019 (UTC)

FULL SUPPORT - YAS, get it!! Just tell me what to do and I'll do it! RuneMetrics icon.png Tyler JarretTalkLight animica.png 18:31, 23 January 2019 (UTC)

Support - I ran into the Value1/Value2/Value3 problem when adding monster combat levels to the quest details template a while ago. {{Get|Dagannoth (Waterbirth Island)#Blood Runs Deep|Combat level}} would be a lot more convenient and future proof than trying to do stuff with Combat level3. "Should make our tables much more efficient - maybe removing the (self-imposed) limit we currently have" would also be pretty cool. Iiii I I I 21:06, 23 January 2019 (UTC)

Support - Even though I don't really work on the backend technical stuff on the wiki I definitely understand the use of this and how helpful it can be. Superiosity the WikianQuick chat button.png : Buying item: Oily fishing rod 23:14, 23 January 2019 (UTC)

Support - As you know I ran into some problems with querying Values from smw. I'm all for a rework. How much data are we going to add to SMW? Am I correct in assuming as much as possible? Also are we going to be able to add the subobjects without breaking to much? I have no idea what all pulls data from SMW and how much of it will break from adding submodules. I'd be for adding a subobject for things without versions as well, it should make querying the data easier. Seers headband 2 chathead.png Elessar2 (talk) 17:33, 24 January 2019 (UTC)

Support/Comment - I dont really work on this backend stuff (much/yet), but recently started learning how modules/lua work on the site. So, I support doing this as long as the people who actually can utilize this support it too (and someone, someday is willing to teach me what it all really means.) Also, it seems like once we initially roll out the subojects/update, certain things will be much more cohesive and easier to understand (ie, not looking like such a scary wall of code). Some time ago, I used to get frustrated when I wanted to change something on a wiki page that looked like a simple table change, only to realize it was templates and modules based, so I just didn't edit for awhile. I would like to suggest that, if possible, the work that is put in to make things function with this new update also include some strategy to smooth out the learning curve for new editors in this manner. Just my 2gp.  RS AdvLogMyles Prower  Talk 18:12, 24 January 2019 (UTC)

Support/Comment As per Superiosity and Myles Prower Attamaris (talk) 22:19, 24 January 2019 (UTC)

Comment/Closure - This is in general stages of being put into effect:

  • Module:Infobox has been extended to allow for subobjects
  • Module:Infobox Item generates a selection of properties in subobjects
  • Module:Infobox Monster generates a selection of properties in subobjects and the new bestiary is in progress to completion
  • Module:DropsLine generates subobjects of the dropped items on monster pages, which Module:Get drop info puts onto item pages (in addition to the expansions from Forum:Drops table adjustments)
  • In-progress and TODO things
    • [[Module:Infobox Bonuses]] is being fully rewritten to use Module:Infobox, making the SMW changes easier
    • The bestiary rework is still in-progress
    • The equipment tables minor rework will follow bestiary and infobox bonuses
    • Several other infoboxes should have this applied too, primarily NPC but there are likely others that would benefit
    • The equivalents of DropsLine and Get drop info for shops and recipes, and potentially other things should be rewritten to use subobjects
    • The now-redundant properties have not yet been deleted.

If you see anything weird with any of these modules/templates, please do not hesitate to report it to me on my talk page, RS:UH, or in Discord. Quest.png Gaz Lloyd 7:^]Events!99s 20:49, 9 February 2019 (UTC)