Looking for information/resouces on the Xiso file spec

[SoftMachine]

I've been working on a python app that's sort of a replacement for extract-xiso, plus some other features like CSO compression/decompression, iso scrubbing, attach xbe generation, and batch processing. I have it done for the most part besides being able to write a new xiso from a game folder, which I'd consider a pretty important thing to be able to do.

This seems like the most complex task extract-xiso handles and the code for that program is honestly pretty hard for me to follow (I'm not fluent in C and it doesn't appear to be super well organized) so I'm looking for some more resources on the xiso format, how it's laid out, constraints, specs, and such.

[Ernegien]

This is probably even less-readable than C to you, but it's well-structured/commented and the author is still around - https://github.com/antangelo/xdvdfs/blob/main/xdvdfs-core/src/layout.rs

Alternatively, you can also probably use the decompiled output (via dotPeek etc.) of XboxDVD.dll bundled in the XDVDMulleterBeta app which is written in .NET, but possibly cleaner than extract-xiso source.

[Bowlsnapper]

9 hours ago, SoftMachine said:

I've been working on a python app that's sort of a replacement for extract-xiso, plus some other features like CSO compression/decompression, iso scrubbing, attach xbe generation, and batch processing. I have it done for the most part besides being able to write a new xiso from a game folder, which I'd consider a pretty important thing to be able to do.

This seems like the most complex task extract-xiso handles and the code for that program is honestly pretty hard for me to follow (I'm not fluent in C and it doesn't appear to be super well organized) so I'm looking for some more resources on the xiso format, how it's laid out, constraints, specs, and such.

I suppose just extracting the ISO would cover most needs, of course. But do you think it would be possible to actively browse the ISO so you can see what's in it before extracting? Or extract individual files?

[sweetdarkdestiny]

1 hour ago, Bowlsnapper said:

I suppose just extracting the ISO would cover most needs, of course. But do you think it would be possible to actively browse the ISO so you can see what's in it before extracting? Or extract individual files?

[Bowlsnapper]

41 minutes ago, sweetdarkdestiny said:

Ah, of course... Disregard that @SoftMachine. Keep up the good work! I'm gonna go crawl under a rock now...

That mullet is fuckin DIESEL.

[Dempsey_86]

And why don't you use repackinator? The latest version has CSO support. No need to reinvent the wheel

 

[Incursion64]

Repackinator already does this and it’s better imho. oh and it’s open source. 

[Bowlsnapper]

4 hours ago, Dempsey_86 said:

And why don't you use repackinator? The latest version has CSO support. No need to reinvent the wheel

 

... Well maybe I will! I still support this project, however.

[Incursion64]

25 minutes ago, Bowlsnapper said:

... Well maybe I will! I still support this project, however.

Nice, the source may be of use/guidance to what your doing.  It's really underappreciated by the group who benefits the most from it right now.  That fact that you can't even talk about it on the MM discord channel is a whole other level of crazy.  I'm 5 or 6 weeks into waiting for a response to the "review" that will  seemingly never happen lol.  I just feel bad for the people putting all this personal time into building something that's been done before because of some weird beef MM has with other groups doing stuff open source and free.  Good luck with what your working on though, at the very least enjoy the journey. 

[Bowlsnapper]

1 hour ago, Incursion64 said:

Nice, the source may be of use/guidance to what your doing.  It's really underappreciated by the group who benefits the most from it right now.  That fact that you can't even talk about it on the MM discord channel is a whole other level of crazy.  I'm 5 or 6 weeks into waiting for a response to the "review" that will  seemingly never happen lol.  I just feel bad for the people putting all this personal time into building something that's been done before because of some weird beef MM has with other groups doing stuff open source and free.  Good luck with what your working on though, at the very least enjoy the journey. 

I believe this post should be directed toward @NeMesiS.  Wish I could take credit though

[Dempsey_86]

so @Bowlsnapper, are you thinking at an xiso build? haha

[SoftMachine]

23 hours ago, Ernegien said:

This is probably even less-readable than C to you, but it's well-structured/commented and the author is still around - https://github.com/antangelo/xdvdfs/blob/main/xdvdfs-core/src/layout.rs

Alternatively, you can also probably use the decompiled output (via dotPeek etc.) of XboxDVD.dll bundled in the XDVDMulleterBeta app which is written in .NET, but possibly cleaner than extract-xiso source.

I'll have a look at that repo but C# is way easier for me to read, that XboxDVD dll is definitely more organized than extract-xiso as well, thanks for the tips

4 hours ago, Incursion64 said:

Repackinator already does this and it’s better imho. oh and it’s open source. 

Repackinator is a great tool, I'm not trying to replace it, I use it personally and I've based quite a few of the things in the program off of it and xbox_shrinker, but I'm using this as a learning tool for myself since I've only started programming a few months ago. I've also built in something similar to xbox_shrinker as a way to recreate redump images with xiso or cso files the program creates, but haven't included a way to calculate and store seeds for non-rc4 discs yet (I'd be surprised if I could get that anywhere near as fast as xbox_shrinker does it, maybe including a dll for that would be better), so having a full sized redump library sitting around wouldn't be necessary for preserving those files. I guess how appealing that is depends on how much storage you have to play with. The program will be open source when it's "complete" or even before then, most of its core functionality is based on open source projects anyway and I don't see a reason not to publish code for something like this.

I did notice there's a lot of work that's been done in the pull requests for the MM stellar-cso repo, but I don't think any have been reviewed, so I'm not surprised Repackinator hasn't been looked at . As far as I can tell, it creates a perfectly compliant cso, the only real difference I've noticed is when/how it handles padding, but the outcome seems to be the same. Idk the specifics of C#'s lz4 library and if it's any different than python's, I'd assume not really.

I also wanted to solve an issue that I've had in the past, which is proper XBE naming for 'HDD ready' games. Although I don't use games stored like that any more, I wanted to make a tool that would just get rid of that issue forever, renaming an entire library based on folder names or actual retail names, or renaming them as they're extracted. Maybe an editor too that can change basic info like title id, region, and naming. Custom title images would be kind of cool but I haven't really explored that, I know there's much more there than just editing some cert bytes.

[Ernegien]

3 hours ago, SoftMachine said:

I'll have a look at that repo but C# is way easier for me to read, that XboxDVD dll is definitely more organized than extract-xiso as well, thanks for the tips

This honestly slipped my mind as I don't really deal with anything other than extract-xiso, but repackinator source would be a way better reference than mulleter decompilation. I'd still refer back to antangelo's project for more context however as repackinator seems overly complex for some of what it's doing and doesn't have any comments so it may be harder for you to follow the logic.

https://github.com/Team-Resurgent/Repackinator/blob/main/Resurgent.UtilityBelt.Library/Utilities/XisoUtility.cs

[Bowlsnapper]

4 hours ago, SoftMachine said:

I'll have a look at that repo but C# is way easier for me to read, that XboxDVD dll is definitely more organized than extract-xiso as well, thanks for the tips

Repackinator is a great tool, I'm not trying to replace it, I use it personally and I've based quite a few of the things in the program off of it and xbox_shrinker, but I'm using this as a learning tool for myself since I've only started programming a few months ago. I've also built in something similar to xbox_shrinker as a way to recreate redump images with xiso or cso files the program creates, but haven't included a way to calculate and store seeds for non-rc4 discs yet (I'd be surprised if I could get that anywhere near as fast as xbox_shrinker does it, maybe including a dll for that would be better), so having a full sized redump library sitting around wouldn't be necessary for preserving those files. I guess how appealing that is depends on how much storage you have to play with. The program will be open source when it's "complete" or even before then, most of its core functionality is based on open source projects anyway and I don't see a reason not to publish code for something like this.

I did notice there's a lot of work that's been done in the pull requests for the MM stellar-cso repo, but I don't think any have been reviewed, so I'm not surprised Repackinator hasn't been looked at . As far as I can tell, it creates a perfectly compliant cso, the only real difference I've noticed is when/how it handles padding, but the outcome seems to be the same. Idk the specifics of C#'s lz4 library and if it's any different than python's, I'd assume not really.

I also wanted to solve an issue that I've had in the past, which is proper XBE naming for 'HDD ready' games. Although I don't use games stored like that any more, I wanted to make a tool that would just get rid of that issue forever, renaming an entire library based on folder names or actual retail names, or renaming them as they're extracted. Maybe an editor too that can change basic info like title id, region, and naming. Custom title images would be kind of cool but I haven't really explored that, I know there's much more there than just editing some cert bytes.

When games are extracted to run off the HDD, does the XBE always look for them in a root folder instead of specifying the optical drive at the beginning of the path, or is that patched somehow?

[Incursion64]

19 hours ago, Bowlsnapper said:

When games are extracted to run off the HDD, does the XBE always look for them in a root folder instead of specifying the optical drive at the beginning of the path, or is that patched somehow?

default.xbe is inside the Xiso along with whatever other folders the game has.  So basically the xbe is pulling whatever assets are along side of it. The Attach file to launch an xiso or compressed file is just mounting the game with the virtual drive.  The XBE naming that's saved to that attach is what you are seeing displayed on your xbox, or what is recorded in the DB for xbmc if you use that. 

 

 

 

 

[Bowlsnapper]

7 hours ago, Incursion64 said:

default.xbe is inside the Xiso along with whatever other folders the game has.  So basically the xbe is pulling whatever assets are along side of it. The Attach file to launch an xiso or compressed file is just mounting the game with the virtual drive.  The XBE naming that's saved to that attach is what you are seeing displayed on your xbox, or what is recorded in the DB for xbmc if you use that. 

 

 

 

 

That's what I figured, is that the XBE considered itself to be in a "root" location and just pulled the files using that as the basis for the structure.

But... I think filenames can be longer on disc and use other characters as well, and are not restricted by the same rules as FATX. If that is true, does that ever cause problems, since I'm sure those would have to be truncated and would cause problems for the XBE trying to use those assets...