replays bug(s)

Open-source port of PoP that runs natively on Windows, Linux, etc.
Falcury
Calif
Calif
Posts: 565
Joined: June 25th, 2009, 10:01 pm

Re: replays bug(s)

Post by Falcury »

David wrote:If I pass it in the command line, then it just prints an error message.
If I don't pass any *.P1R filename then it crashes.
Strangely, for me it doesn't crash (the title screen just reloads). :?
I'll try to look into it further later today. Any idea where it segfaults?
Falcury
Calif
Calif
Posts: 565
Joined: June 25th, 2009, 10:01 pm

Re: replays bug(s)

Post by Falcury »

I was able to reproduce the crash, after realizing you meant pressing Tab immediately after loading an incompatible replay directly (double-clicking, etc.) had failed. I tried to fix it here:
https://github.com/NagyD/SDLPoP/pull/114

There were a couple really stupid little bugs :oops: I only found them after using a debugger to look for problems systematically...
Hopefully it no longer crashes?
David
The Prince of Persia
The Prince of Persia
Posts: 2846
Joined: December 11th, 2008, 9:48 pm
Location: Hungary

Re: replays bug(s)

Post by David »

Falcury wrote:I tried to fix it here: https://github.com/NagyD/SDLPoP/pull/114 [...] Hopefully it no longer crashes?
Thanks, it seems to fix the crash!
User avatar
oitofelix
Wise Scribe
Wise Scribe
Posts: 227
Joined: February 17th, 2016, 1:59 pm
Location: Brazil
Contact:

Re: replays bug(s)

Post by oitofelix »

As I announced here, replays have been added to MININIM. For this feature in action see here. This post is intended to describe MININIM's replay file format and give a rationale for it.

To begin with, MININIM accepts the following mutually consistent premises regarding replays:
  1. Replay files purpose is primarily to allow for record, playback, sharing and mechanical verification of solutions to level sets on a level per level basis, like video walkthroughs, additionally allowing external chaining for playback and verification of a solution to an entire level set. Therefore a given replay file must start at the very first cycle of a level and must not extend beyond the very end of that same level. Replays can be incomplete, though, meaning that they may end prematurely (useful for showing how not to solve a level ;)).
  2. Replay files are intended to be shared widely across space and time, therefore they must be architecture independent (wideness and endianness), and release/build/version agnostic.
  3. Replay files are only meaningful along with their respective level sets, but they should not replicate the information thereof, since it's assumed all parts interested in a given replay have access to the level sets and replays should stand against the actual level set, not a slightly modified (potentially unoticeably different) version.
  4. Replay files should carry the absolute minimum information about the engine state (usually only what can be provided by user configuration), and the engine must ensure it can reliably record and reproduce the derivative states in all applicable situations, not allowing the user to interfere in any way with the results (by subsequent reconfiguration of the engine using command line or key bindings, let's say). Furthermore, it should be transparent to everybody the effective configuration of a replay, as a way to compare the equivalence of solutions and prevent inadvertent cheating.
  5. Replay files should have the minimum overhead possible to accomplish the aforementioned goals.
MININIM replay file format is as follow:
  • Signature (string, 14 bytes): "MININIM REPLAY"
  • File format version (uint32_t). Currently 1.
  • Packed boolean engine configuration (uint32_t): only two bits currently used, the remaining are reserved for future backwards-compatible extensions.
    • Bit 0: mirror level mode
    • Bit 1: immortal mode
  • Movements enumeration (uint32_t): current valid values are NATIVE (0) and LEGACY (1), but MININIM may support other sets of movements in the future.
  • Semantics enumeration (uint32_t): current valid values are NATIVE (0) and LEGACY (1), but MININIM may support other sets of semantics in the future.
  • Start level (uint32_t): recorded level.
  • Start time (uint32_t): time count at level start.
  • Time limit (uint32_t): time limit for finishing the level set
  • Total lives (uint32_t): number of life containers at level start
  • KCA (uint32_t): kid's counter attack skill at level start
  • KCD (uint32_t): kid's counter defense skill at level start
  • Random seed (uint32_t): random seed which fully replicates the level state.
  • Packed gamepad state (uint8_t[]): array that extends until the end of the replay file, containing one byte per time cycle having the following format: (notice that the two higher bits of the byte are reserved for backwards-compatible future extensions).
    • Bit 0: up command
    • Bit 1: down command
    • Bit 2: left command
    • Bit 3: right command
    • Bit 4: shift command
    • Bit 5: enter command
Notice that all fields are 32 bits wide, even though the info currently stored don't necessarily require that much space. They amount for a constant overhead (independent of the number of cycles in the replay) and could later on be split to accommodate for backwards-compatible extensions.

A replay that completely solves its respective level is said to be "complete". Replay files may be chained (in increasing level order) to present a solution to an entire level set. In this case, such replay chain is said to be "complete" if and only if each of its members is complete. It's said to be "valid" if and only if each of its sequent pairs are valid. The precise definition of validity for a pair of consecutive replays is: the pair of replays A-B is valid, if and only if, initial --start-level of B is equal to the the final one of A, initial --start-time of B is greater or equal to the final one of A, --time-limit, --total-lives, --kca and --kcd of B are all respectively less or equal to the final ones of A. Notice that computing the validity of the replay pair A-B requires fully replaying A to access its final outcome, because that information (purposely) is not available in the replay file itself. A replay chain that is complete, valid and has one replay for each level in a level set (usually 14) is a authentic solution to its respective level set. MININIM can mechanically check for these, and this checking is designed to be easily scriptable.

As you may have noticed, MININIM's replay format, unlike SDLPoP's format:
  1. Doesn't have meta-information like player name or creation time. These are unessential, and may be stored in other way.
  2. Doesn't specify level set specific information. The maxim is: if MININIM is set up for play, it's set up for replay.
  3. Doesn't specify video/audio resources information. Graphics, audio, levels and replay data are all orthogonal to each other. It should be the user's choice what combination to use; the way originally intended by the modder or otherwise.
  4. Doesn't include save state information. Including such data is at odds with all of MININIM's 5 aforementioned replay premises.
    1. It's unnecessary, as all replays must by principle start at the level beginning. Furthermore, this is what people usually want anyway. Allowing for recording to start anywhere in a level makes synchronization with the exact level start (needed for complete replays) harder from a user standpoint.
    2. Save states are inherently non-portable. It's prohibitively difficult to make save state information shareable across different sets of architectures (wideness and endianness), after all they are basically a memory snapshot (a very architecture dependent thing), and it's difficult to ensure stability across different releases --- a relatively minor change to the engine may have profound implications on how save states are represented.
    3. The level set is entirely obviated by the save state, which replicates its information, without guarantee of exact correspondence (unless further checking steps have been implemented, which will require the original level set anyway).
    4. Save states carry just too much information about the engine state. Only basic information should be required and desired because replays are just "mechanical" players whose purpose is to prove whether and how a level has been solved, under a few barebones initial conditions agreement. Save states can't be transparent as replays are intended to be.
    5. Save states are relatively big when compared to raw replay data, adding unneeded overhead to the format.
  5. Doesn't exclude the possibility of movements using two directions in the same axis. MININIM implements smooth command-movement transition, and heavily uses such feature for the player's convenience. As I understand it that would be impossible using SDLPoP's move encoding.
Finally, two side notes:
  1. I'd like to say that, although initially I thought it was betters for implementations to use a standard format, I got convinced later on that there is no point, since they can't usefully share replays and a common specification has to be very generalist, to the point of burdening developers too much with alien details.
  2. Although I've initially proposed a generalist approach to replays (by recording all input information), considering this more closely made me realize numerous challenging issues to be solved and wasn't at all clear whether the huge amount of work invested would really pay off, taking into account the needs of the PoP modding community (which gave me the basis to conceptualize MININIM's 5 replay file format premises).
Thanks for listening. I look forward to your thoughts on this.
Last edited by oitofelix on January 6th, 2017, 4:25 am, edited 1 time in total.
Bruno Félix Rezende Ribeiro (oitofelix)
MININIM author
User avatar
Norbert
The Prince of Persia
The Prince of Persia
Posts: 5743
Joined: April 9th, 2009, 10:58 pm

Re: replays bug(s)

Post by Norbert »

Is there anything I need to change at popot.org other than accepting *.mrp files in addition to *.p1r files?
I'm already planning to have the website say whether available replay files are for SDLPoP or MININIM.

Best regards,
Norbert
User avatar
oitofelix
Wise Scribe
Wise Scribe
Posts: 227
Joined: February 17th, 2016, 1:59 pm
Location: Brazil
Contact:

Re: replays bug(s)

Post by oitofelix »

Norbert wrote:I'm already planning to have the website say whether available replay files are for SDLPoP or MININIM.
Initially I thought it'd be useful to provide a "Finishable in MININIM" (for each legacy mod) read-only checkbox. Thinking about it now, makes me realize having a valid and complete replay chain is sufficient for knowing that. Therefore, what you propose should be enough, because anyone finishing a previously never-finished-in-MININIM mod should record a replay chain and submit it as a proof of his accomplishment.

Replays also have embedded any MININIM configuration switch needed to make the level set finishable, therefore it isn't even necessary to tell people how to invoke MININIM to play that particular mod. (it may be convenient, though)

Are you planning to script replay chain validation on submission? In case you want to deterministically validate the format of submitted files (instead of relying on file extension or likewise heuristics), --replay-info does the job. It returns error status 0 for valid replay files, non-zero otherwise. Of course, truly validating the replay chain by checking it solves the level set is ideal, but costly.
Bruno Félix Rezende Ribeiro (oitofelix)
MININIM author
User avatar
Norbert
The Prince of Persia
The Prince of Persia
Posts: 5743
Joined: April 9th, 2009, 10:58 pm

Re: replays bug(s)

Post by Norbert »

oitofelix wrote:Are you planning to script replay chain validation on submission?
No. On the client side I only use the accept attribute of (file's) <input> to filter for *.p1r (and *.mrp, soon). On the server side I also check this extension, plus the file size (there's a max), and - soon - the file signature. This should be enough, considering uploading requires a user account. In the unlikely event that something unwanted nonetheless comes through, I will remove it by hand and if necessary ban IPs. The website('s e-mail service) notifies me about all uploads.
User avatar
oitofelix
Wise Scribe
Wise Scribe
Posts: 227
Joined: February 17th, 2016, 1:59 pm
Location: Brazil
Contact:

Re: replays bug(s)

Post by oitofelix »

Norbert wrote:The website('s e-mail service) notifies me about all uploads.
Will upload of incomplete replay chains be allowed? Will a single user be able to submit multiple replay files (for a complete chain, let's say)? Will submitted replays be grouped by author? I don't know if SDLPoP supports single replay files that span several levels across. As explained MININIM, by principle, doesn't. If such feature turns out to be desirable, MININIM would probably implement a container format to encapsulate regular replay files (possibly compressed).
Bruno Félix Rezende Ribeiro (oitofelix)
MININIM author
User avatar
Norbert
The Prince of Persia
The Prince of Persia
Posts: 5743
Joined: April 9th, 2009, 10:58 pm

Re: replays bug(s)

Post by Norbert »

Mod pages have an area for submitting replays. The area contains a drop-down preceded by the label "Level:", and its options are:
Spoiler: show
lvl 1
lvl 2
lvl 3
lvl 4
lvl 5
lvl 6
lvl 7
lvl 8
lvl 9
lvl 10
lvl 11
lvl 12a
lvl 12b
princess
potions
demo
other
lvl 1 alt.
lvl 2 alt.
lvl 3 alt.
lvl 4 alt.
lvl 5 alt.
lvl 6 alt.
lvl 7 alt.
lvl 8 alt.
lvl 9 alt.
lvl 10 alt.
lvl 11 alt.
lvl 12a alt.
lvl 12b alt.
princess alt.
potions alt.
demo alt.
other alt.
In addition to this, there is an input field labeled "Description:", that can optionally be used.

The assumption is that users will upload per level, not spanning multiple levels across. The drop-down options also steer users towards this. Most likely complete level runs, but partial runs are possible, for instance to show something specific within the level. If the alternative ("alt.") or other ("other", "other alt.") options are used, the description field becomes more relevant.
oitofelix wrote:Will submitted replays be grouped by author?
Currently, replays are sorted in the order of the drop-down above. If, for example, two users each upload a replay file for "lvl 1", these files end up at the top of the overview. The rationale is that visitors are looking for replays of specific levels and are less interested in the authors. I don't expect the replay lists to ever become too large to scroll or Ctrl+f search through, but if they do I can add an option for alternative sorting, e.g. by author.

From what I've seen, players usually record playthroughs of levels if none are available on YouTube. I can imagine the same will apply to replay files. Some exceptions here and there, for instance if someone wants to share a faster run, or an alternative route, things like that. Each user can only have 1 replay for each option of the drop-down list; as it says in the upload area, additional uploads for the same option means overwriting an old replay.
User avatar
oitofelix
Wise Scribe
Wise Scribe
Posts: 227
Joined: February 17th, 2016, 1:59 pm
Location: Brazil
Contact:

Re: replays bug(s)

Post by oitofelix »

Norbert wrote:The assumption is that users will upload per level, not spanning multiple levels across.
Interestingly enough we independently came to the same conclusion. ;)
Norbert wrote:Currently, replays are sorted in the order of the drop-down above. If, for example, two users each upload a replay file for "lvl 1", these files end up at the top of the overview. The rationale is that visitors are looking for replays of specific levels and are less interested in the authors. I don't expect the replay lists to ever become too large to scroll or Ctrl+f search through, but if they do I can add an option for alternative sorting, e.g. by author.
In principle,one might be looking not only for a complete, but also valid replay chain, though. MININIM's option --validate-replay-chain may help them to obtain one in case they randomly select replay files (one from each author, let's say); (it must be possible to modify replay initial parameters to validate the entire chain, for this to succeed). That's what that option is designed for.

PS: By the way, I think the standard legacy level set deserves to be listed among mods (although not being strictly a mod, albeit certainly a level set), because people may want to submit their replays for it. I'd submit MININIM's official ones.
Last edited by oitofelix on January 6th, 2017, 5:16 pm, edited 1 time in total.
Bruno Félix Rezende Ribeiro (oitofelix)
MININIM author
User avatar
Norbert
The Prince of Persia
The Prince of Persia
Posts: 5743
Joined: April 9th, 2009, 10:58 pm

Re: replays bug(s)

Post by Norbert »

I think I'll leave it up to downloaders to optionally verify the integrity of replays. What I'll do is add an "i" icon next to the "replays:" text at the top, with a link to a page that explains what replays are. On that page I will mention MININIM's validate option, among other things.

I have also added the original Prince of Persia here. Note that replay uploading is still in beta, and in the near future - after modification of the website - everything replay-related will be reset (including the deletion of uploaded replays).
User avatar
oitofelix
Wise Scribe
Wise Scribe
Posts: 227
Joined: February 17th, 2016, 1:59 pm
Location: Brazil
Contact:

Re: replays bug(s)

Post by oitofelix »

Norbert wrote:What I'll do is add an "i" icon next to the "replays:" text at the top, with a link to a page that explains what replays are. On that page I will mention MININIM's validate option, among other things.
Nice.
Norbert wrote:I have also added the original Prince of Persia here.
Thanks. :) PS: I just uploaded a replay for level 1, but things didn't work quite as expected. I guess it's not ready yet. :P The replay is listed there, but can't be downloaded (404 error), and it was renamed to have the p1r extension.
Norbert wrote:Note that replay uploading is still in beta, and in the near future - after modification of the website - everything replay-related will be reset (including the deletion of uploaded replays).
I don't see why MININIM replays have to be deleted. Could you explain?
Bruno Félix Rezende Ribeiro (oitofelix)
MININIM author
User avatar
Norbert
The Prince of Persia
The Prince of Persia
Posts: 5743
Joined: April 9th, 2009, 10:58 pm

Re: replays bug(s)

Post by Norbert »

oitofelix wrote:replay is listed there, but can't be downloaded (404 error)
Can you send me the file via private message, then I will test locally.
oitofelix wrote:I don't see why MININIM replays have to be deleted. Could you explain?
The reason is that the overviews are not created based on available files, but the other way around. (As you just noticed. ;)) And the relevant table still needs to undergo several changes that would make it too much a hassle to update old data.
User avatar
oitofelix
Wise Scribe
Wise Scribe
Posts: 227
Joined: February 17th, 2016, 1:59 pm
Location: Brazil
Contact:

Re: replays bug(s)

Post by oitofelix »

Norbert wrote:Can you send me the file via private message, then I will test locally.
Those are the ones at the top of their respective footage in the MININIM's standard legacy level set walkthrough post, specifically 01.mrp.
Norbert wrote:The reason is that the overviews are not created based on available files, but the other way around. (As you just noticed. ;)) And the relevant table still needs to undergo several changes that would make it too much a hassle to update old data.
I see.
Bruno Félix Rezende Ribeiro (oitofelix)
MININIM author
User avatar
Norbert
The Prince of Persia
The Prince of Persia
Posts: 5743
Joined: April 9th, 2009, 10:58 pm

Re: replays bug(s)

Post by Norbert »

I just tried to upload the file in the production environment (the live website), and for me it works.
Can you tell me what exactly you did? Also, can you retry adding it.
Post Reply