What's new
That is unexpected, given the tool was made on Win7x64, doesnt have any external dependencies. I did not write much error checking given its a quick dirty tool for a one time task. Directories are created using a canned function 'ForceDirectories' that creates the child and parent directories in one task; so it doesn't make sense why creating the root 'crysking' folder makes it work. Any chance you were running this in the root of a drive? Windows doesnt allow non-admin privileged programs to create root folders.

I was also disappointed listening to the crysking music, but the topbladv ones are pretty rad and not sampled (I dont think?).
 
I'm not in a root, no. Eh, it works when I make the directory myself. *shrug*. I'm in an administrator command window too. Could be something with my system.

Anyway yeah Top Blade music is pretty sweet!! Legit tracked for the most part that I've seen so far. Stage1.s3m is on fire! There's one track (intro) that's all sampled except for tracked drums, but the sampled parts are at like 18khz so they sound much better. Crystal of kings sounds like it's coming through an old school telephone.
 
I noticed the sound when I looked at a video of Crystal of Kings - I thought it's the emulator or recording...
Who knows, I can think of many reasons why things like this happen... maybe the music was produced before all hardware and tools were finished, and they were originally planning an MP3-style audio system? That is disappointing, but sometimes you get things like this when you have only limited time and money...

Who is going to make the first modded game? :)
 
It's probably worth taking a few existing S3M files as a test first - two of the reasons why they could've done it like this are RAM limitations, or if the initial version of the S3M player was buggy and didn't play correctly.
 
So, like, @brizzo we’re gonna need a recompiler now?? ;)
That is definitely something we can explore. I have not fully solved all the values that are used in the file system. When you run the fs extract tool, in the log you see a value 'Unknown'; I am not sure what this value represents.

ROM Structure:

0x0000 - 0x0013 = MESG dump file v2.5<0x00>
0x0014 - 0x006F = Filled with <0xCC>
0x0070 = uint32_t file_count;
0x0074 = uint32_t total_data; // total data from 0x0000 to end of last file
0x0078 - 0x007F = Filled with <0xCC>

0x0080 = start of record index, struct { uint32_t file_offset; uint32_t unknown_value; } // repeats for file count

file_offset points to file record, consisting of:
filename string, which is <0x00> terminated and ends on 4 byte alignment (some strings are followed by 1-3 bytes of bad/random data)
uint32_t file_size;
<file data>, 4 byte alignment (some entries have 1-3 bytes of bad/random data before next record starts)

In summary, we just need to solve the 'unknown_value' in the initial record table, otherwise we have all the information to compile files back into the rom file system :)

PS: In the log file generated by bsc_fs_extract tool, the 'offset' is actually the start of <file data>, not the file_offset which points to the start of the record, in case anyone tries analyzing these values for some correlation
 
(I had the same problem that I had to create the base directory first.)

I had a look at top blade: if you sort by filename then the last byte is frequently the same for files with the same path length in the same directory. You get a similar grouping if you sort by the second byte. So maybe two 16 bit values?
 
I was able to figure out what the unknown field is by debugging/disassembling the bios with mame. Its a 32bit hash/crc of the filename. The following code can be used to generate the hash/crc values.
C:
#include <stdio.h>
#include <string.h>
#include <stdint.h>

int main(int argc, char **argv) {
  int i;
  uint32_t r0;
  uint32_t r1;
  uint32_t r2;
  uint32_t r4 = 0;
  uint32_t crc = 0;


  if(argc < 2) {
    printf("bcs-filename-crc <filename string>\n");
    return 1;
  }

  for(i = 0; i < strlen(argv[1]);i++) {
    crc = crc << 8;
    crc += argv[1][i];

    r0 = 0xfffffd;
    r1 = 1;

    while(r0 < crc) {
      r0 = r0 << 1;
      r1 = r1 << 1;
    }

    while(r1 > 0) {
      if(r0 < crc) {
        crc = crc - r0;
      }
      r0 = r0 >> 1;
      r1 = r1 >> 1;

    }
    r4 = r4 ^ argv[1][i];
  }

  crc = r4 << 24 | crc;
  printf("FILENAME CRC: %08x\n", crc);
}

Code:
$ ./bcs-filename-crc 'SND\VOICE\TOPBLADEV.RAW'
FILENAME CRC: 36d5673e
Couple things to note.
  • I believe you will always want to use uppercase for filenames/paths
  • When generating the filesystem, I believe the order of the files in the header need to be sorted by the crc/hash values
  • When running this via the command line be sure to put single quotes around the filename or the shell will likely think "\" is to escape the next char
 
@ack thank you so much for taking the time to solve this! You are the real MVP!! I started looking at this yesterday in MAME debugger, then pondered that I have bigger problems to solve before visiting editing game content :D

Do you have an opcode reference or manual for the SE3208 instruction set? I looked around a bit but didn't find anything specific
 
@ack thank you so much for taking the time to solve this! You are the real MVP!! I started looking at this yesterday in MAME debugger, then pondered that I have bigger problems to solve before visiting editing game content :D

Do you have an opcode reference or manual for the SE3208 instruction set? I looked around a bit but didn't find anything specific
I didn't have any luck finding a manual and was going off whats in mame code for the cpu

https://github.com/mamedev/mame/tree/master/src/devices/cpu/se3208

I was a bit confused at first with the register names in the debugger/disassembler

SUB %SR2,%SR0,%DR2

It appears that "S" is signifying source register(s), and "D" is the destination register. So that is really %R2 - %R0 => %R2. The disassembly didnt make any sense until I figured that out.

As for tracking down the code in the bios, if anyone cares. I did a strings on the bios rom and noticed junk about the filesystem at the end, which included a reference to "binlist.ini". That file exists in every game's filesystem, and seems to be the initial file the bios loads from the filesystem. Inside that file seems to be the filename of the actual game binary that gets loaded. Checking the unknown value for BINLIST.INI in each of the game roms showed they were all 270af723. Based on that I setup a register point on all the registers to trigger on that value. Once that triggered it gave me a PC location for which I could work backwards from to figure out the crc/hash logic.
 
SUB %SR2,%SR0,%DR2

It appears that "S" is signifying source register(s), and "D" is the destination register. So that is really %R2 - %R0 => %R2. The disassembly didnt make any sense until I figured that out.
This is definitely super helpful! I did not understand this, and my brain still isn't understanding LERI either :D

Checking the unknown value for BINLIST.INI in each of the game roms showed they were all 270af723. Based on that I setup a register point on all the registers to trigger on that value. Once that triggered it gave me a PC location for which I could work backwards from to figure out the crc/hash logic.
Good idea working from this direction. @Fluffy was on the right track speculating the unknown value was related to filenames :)


In other good news, the PCBs were delivered today but wasn't able to assemble and test until after midnight tonight. Won't have a lot of time until middle of next week to invest more time, but I got started! Programmed the flash with multiple roms before soldering. With new pcb designs I start bring up with the bare essentials in case any issues need to be solved. Right now it has usb power and cart vcc filter, pull up resistors for pic security signals, cpld for logic to control the flash, and a single 1gigabit flash chip.

Things are looking good so far! Started with trying to boot crysking, no luck.... but I tried topbladv and it booted up! Also have donghaer and officeye on the flash, going to test those now and see what I can learn.

20190713_014833.jpg


:thumbsup:
 

Attachments

  • 20190713_014916.jpg
    20190713_014916.jpg
    154.9 KB · Views: 179
Very cool indeed! If I can add Top Blade and Dongul to my arsenal, I’ll be happy!
 
Things are looking good so far! Started with trying to boot crysking, no luck....
You might want to also NOP writes to the flash chip. Commands sent are only valid for the Intel E28F128J3A chips:
Read Array SCS/BCS 1 Write X 0xFF 1
Read Identifier Codes SCS/BCS = 2 Write X 0X90 Read IA ID 1,7

I also found a test branch which I think is related to chip ID that is never taken by MAME (replacing the JNZ by a JMP might help too).
But I have patched it "on the fly" and haven't noted exact address. Should be simple enough to find again.

BTW my patched topbladv has also been confirmed working on real hardware:

T92MaEA_d.jpg
 
Things are looking good so far! Started with trying to boot crysking, no luck....
You might want to also NOP writes to the flash chip. Commands sent are only valid for the Intel E28F128J3A chips:Read Array SCS/BCS 1 Write X 0xFF 1
Read Identifier Codes SCS/BCS = 2 Write X 0X90 Read IA ID 1,7

I also found a test branch which I think is related to chip ID that is never taken by MAME (replacing the JNZ by a JMP might help too).
But I have patched it "on the fly" and haven't noted exact address. Should be simple enough to find again.

BTW my patched topbladv has also been confirmed working on real hardware
Electronics on carpet!? Oh noes, how will the internet react? ;)

Awesome to see this project progressing so quickly
 
Back
Top