Oolite Wiki

[maik]

I wonder how we could automate this. Long-term it is a pain to maintain things twice, and thus will not happen consistently (e.g. if an OXZ author updates the description for the extension manager the copy on the wiki would need an update, too). Couldn't we have one source for this data only, and have both generated from this?

I thought of something like automatically generating wiki pages from expansions. This could be a small batch job that runs at regular intervals or maybe as part of the CI/CD pipeline.

• it starts off with a list of expansions - this could be the same source that the expansion manager[ uses. (see http://wiki.alioth.net/index.php/Expansions_Manager)

• From that list it uses the download links, then downloads and scans the OXPs, OXzs and whatever it can get hold of. Not sure here, currently I think reading the manifest could be sufficient, future versions could access more information.

• From this information we could then generate wiki pages, one or several if need be. Or filterable/sortable ones if that is possible.

• Finally these pages get uploaded into the oolite wiki.

Not that hard to code. It would more require some knowledge about the expansions and the source to read and the destination to write.

I agree the initial creation of wiki pages from manifests is not that hard to code, you just need a unique identifier across manifests and the wiki to not start creating duplicates.

The more difficult bit comes when there are updates. One author might leave the manifest as is and just update the wiki page, another might update the manifest but does not like editing the wiki, and the third one is happy to keep both in sync

I believe we should have a complete strategy before we implement such automation. The risk to mess up the wiki is quite real

I can see if I can create a wiki playground again for testing when we get somewhat further.

[hiran]

I agree the initial creation of wiki pages from manifests is not that hard to code, you just need a unique identifier across manifests and the wiki to not start creating duplicates.

The more difficult bit comes when there are updates. One author might leave the manifest as is and just update the wiki page, another might update the manifest but does not like editing the wiki, and the third one is happy to keep both in sync

I believe we should have a complete strategy before we implement such automation. The risk to mess up the wiki is quite real

For updates you mention different cases:

• Automatically generated pages: This is trivial as for updates we can simply overwrite the page.

• Some authors just update the wiki page: We could track for manual changes and no longer auto-update the page - but this behaviour would mean once touched it is back in manual mode, and the quality would deteriorate. Therefore I propose to discourage manually updating pages that were generated or even protect them from interactive users.

• Those that update the manifest only are the good boys, and the first rule applies.

• A bit stranger is if users keep both manifest and wiki page up to date. This is the only case where I see a strategy is necessary. But once we have the automation ready a goal should be set to synchronize the data on the manually maintained wiki pages into the OXPs so we have one source of truth. OXP maintainers should actually agree as from then on they only would have to care about their OXP and the inclusion in the OXP list but nothing else. it eases their life.

I can see if I can create a wiki playground again for testing when we get somewhat further.

This is actually the best approach to move on.

[hiran]

In the meantime I could give that batch job a try.
Who knows which download URL is used by the Expansion Manager to get the most up to date list of plugins?

I found this line so far:
https://github.com/OoliteProject/oolite ... ager.m#L48

which indicates to me this would be the URL used by oolite to find expansions:
http://addons.oolite.org/api/1.0/overview

Next is to figure out syntax and grammar of the data. Or the best parser to choose, as I'd prefer to not implement that myself.
However I am not familiar with Objective C, yet it seems the data format is whatever is generated using the NSCoding protocol.

[hiran]

Looking at https://nshipster.com/nscoding/, it seems NSCoding can emit SQLite, XML, or NSData formats.
Would it be possible to receive the data from http://addons.oolite.org/api/1.0/overview as XML?

Then the other steps to analyze data and reformat it into wiki pages would be one simple XSLT step - independent of the chosen programming language.

[hiran]

Ok, I am two steps further now.

I created a parser for this strange data format and have an application that can now parse http://wiki.alioth.net/index.php/Torus_Drive, transform it into a W3C DOM and transform it using XSLT. Now I have something like an automatic index of expansions.

Then I went further and extracted the download_url information to download and cache all the OXPs in a local directory. They also get checked whether they are valid ZIP files and contain a manifest.plist.

Interestingly I now have a description (usually a brief summary) from the overview, and there is another description in the OXPs manifest that looks similarly brief. So no much gain.

Where would an OXP contain further information? Or should the check simply be whether there is a valid information_url?

[Cholmondely]

Ok, I am two steps further now.

I created a parser for this strange data format and have an application that can now parse http://wiki.alioth.net/index.php/Torus_Drive, transform it into a W3C DOM and transform it using XSLT. Now I have something like an automatic index of expansions.

Then I went further and extracted the download_url information to download and cache all the OXPs in a local directory. They also get checked whether they are valid ZIP files and contain a manifest.plist.

Interestingly I now have a description (usually a brief summary) from the overview, and there is another description in the OXPs manifest that looks similarly brief. So no much gain.

Where would an OXP contain further information? Or should the check simply be whether there is a valid information_url?

Look at the note I wrote to you on the Village Pump on the wiki: http://wiki.alioth.net/index.php/EliteWiki:Village_pump

[hiran]

Look at the note I wrote to you on the Village Pump on the wiki: http://wiki.alioth.net/index.php/EliteWiki:Village_pump

Thank you for pointing that out. Well, if we want to autogenerate pages/reports from OXPs we should look at the information within the OXPs. Meanwhile I stumbled over a few OXPs with <expansion_name>_README.txt inside. That could be used - but it is just a plain old ASCII file for humans to read - no markup inside. We may want to top that up, e.g. allowing HTML.

On the other hand I am tearing my hair on the manifest.plist data format. See http://wiki.alioth.net/index.php/Manife ... tom_Format
- inside a dictionary, do we have quoted or unquoted values?
- inside a dictionary, do we have quoted or unquoted keys?
- do we allow/need colons or/and equals characters between key and value?
- does the 'key'='value' always end with a semicolon or is that just required to separate them?
- in lists, do we always need commas or just as a separator between values? How come some lists end with a comma?

Somehow I hope this is all not intentional and noone checked these typos. And then I am wondering how all these OXPs work in Oolite today...

[another_commander]

On the other hand I am tearing my hair on the manifest.plist data format. See http://wiki.alioth.net/index.php/Manife ... tom_Format
- inside a dictionary, do we have quoted or unquoted values?
- inside a dictionary, do we have quoted or unquoted keys?
- do we allow/need colons or/and equals characters between key and value?
- does the 'key'='value' always end with a semicolon or is that just required to separate them?
- in lists, do we always need commas or just as a separator between values? How come some lists end with a comma?

Somehow I hope this is all not intentional and noone checked these typos. And then I am wondering how all these OXPs work in Oolite today...

You may want to have a look at these links for a general description of the GNUstep property list grammar:
http://www.gnustep.it/nicola/Tutorials/ ... ode29.html
http://wiki.gnustep.org/index.php/Property_Lists

[Cholmondely]

Thinking of re-organising the Oolite Main Page http://wiki.alioth.net/index.php/Oolite_Main_Page, now that Hiran has started tweaking it!

It currently features the following order of boxes/links:

Code: Select all

Instruction Manual                      Expand Oolite via OXPs

FAQ                                     Oolite Discussion Forum

Ships                                   Stations

Missions                                Equipment

Planet List                             Powers & Organisations

                  Oolite Fiction

I'd like to add a link to the OXZ list on the Oolite.org website and reorganise it in what I hope is a more logical fashion, with Ships and Equipment closer together, etc.

Code: Select all

Instruction Manual                     Expand Oolite via OXZs

FAQ                                    Expand Oolite via OXPs

Ships                                  Planet List

Equipment                              Stations

Missions                               Powers & Organisations

Oolite Fiction                         Oolite Discussion Forum

Any comments?

[Cody]

My fiction seems to be missing from the Oolite Fiction page, Cholly. You could perhaps add An Ideal Victim to it as well - it and its thread contain hints on how to cross the Great Rift in the seventh octant.

[Cholmondely]

My fiction seems to be missing from the Oolite Fiction page, Cholly. You could perhaps add An Ideal Victim to it as well - it and its thread contain hints on how to cross the Great Rift in the seventh octant.

Delighted!

[hiran]

You may want to have a look at these links for a general description of the GNUstep property list grammar:
http://www.gnustep.it/nicola/Tutorials/ ... ode29.html
http://wiki.gnustep.org/index.php/Property_Lists

Good catch! I was searching for something like that.
But there remain questions.

As can be seen, each key-value pair is separated by a semi-colon. Within the pair, the key is seperated from the value with an "equals" (=) sign. The key name is arbitrary, and not put in inverted commas (""). Shown above are: an unknown type (could be some sort of string - TODO), a string, an array, a number and another dictionary (respectively).

The example they show has more semi-colons than the amount required for separation.
Plus if this were authorative, quite a chunk of OXPs the Expansion Manager works with would be syntactically wrong - even the Oolite documentation shows wrong examples at http://wiki.alioth.net/index.php/Manife ... uired_keys. The keys are all quoted.

While creating a parser with all such exceptions is not impossible (actually I just did it) I am still wondering how all these nice OXPs actually work in Oolite. It must have a very forgiving parser, and I hope to meet the same kind of forgiveness...

[hiran]

Thinking of re-organising the Oolite Main Page http://wiki.alioth.net/index.php/Oolite_Main_Page, now that Hiran has started tweaking it!

It seems I skipped some approval process. I am sorry and will discuss ideas here first.

I'd like to add a link to the OXZ list on the Oolite.org website and reorganise it in what I hope is a more logical fashion, with Ships and Equipment closer together, etc.

Code: Select all

Instruction Manual                     Expand Oolite via OXZs

FAQ                                    Expand Oolite via OXPs

Ships                                  Planet List

Equipment                              Stations

Missions                               Powers & Organisations

Oolite Fiction                         Oolite Discussion Forum

Any comments?

Well as a newcomer I felt OXPs as a heading not very talkative. Hence I changed it.
If OXPs and OXZs shall be supported, why not keep that as a detail and not add it to the main page? So the main page would indicate a user that Oolite can be expanded so he/she/it knows when to click that link. In the following page it can still be mentioned that there are two mechanisms with an explanation when to choose which.

Maybe the Ships, Planet List, Equipment, Stations, Missions and Powers stuff could be moved into an 'Encyclopedia Galactica' that explains the simulated universe, while the other topics would set the frame how to install and operate the simulator itself that allows accessing the simulated universe.

[Cody]

Delighted!

I'm obliged, your wizardliness!

[Cholmondely]

Thinking of re-organising the Oolite Main Page http://wiki.alioth.net/index.php/Oolite_Main_Page, now that Hiran has started tweaking it!

It seems I skipped some approval process. I am sorry and will discuss ideas here first.

I'd like to add a link to the OXZ list on the Oolite.org website and reorganise it in what I hope is a more logical fashion, with Ships and Equipment closer together, etc.

Code: Select all

Instruction Manual                     Expand Oolite via OXZs

FAQ                                    Expand Oolite via OXPs

Ships                                  Planet List

Equipment                              Stations

Missions                               Powers & Organisations

Oolite Fiction                         Oolite Discussion Forum

Any comments?

Well as a newcomer I felt OXPs as a heading not very talkative. Hence I changed it.
If OXPs and OXZs shall be supported, why not keep that as a detail and not add it to the main page? So the main page would indicate a user that Oolite can be expanded so he/she/it knows when to click that link. In the following page it can still be mentioned that there are two mechanisms with an explanation when to choose which.

Maybe the Ships, Planet List, Equipment, Stations, Missions and Powers stuff could be moved into an 'Encyclopedia Galactica' that explains the simulated universe, while the other topics would set the frame how to install and operate the simulator itself that allows accessing the simulated universe.

Hiran, I don't usually discuss ideas first. If it is a minor tweak, or including something important, I just do it.

I'm consulting here about a major rejig to the page because I may well have missed something - or somebody may have a better idea than mine.