| Author: | Kevin (guest: search) |
|---|---|---|
| Date: | Fri, Sep 03rd, 2010 @ 05:59 ( . ) |
I recently got a bit more serious about learning 6510 assembly and other aspects of C-64/1541 programming. I was wondering what tools everyone here uses to do the following: (for reference, I've found various programs on the web, but most either don't work with g64 or don't debug like a native C64 app) o edit/copy/visualize the various disk images (particularly .g64 since it contains copy protection data) o step through the memory of the C-64 as a program is running (debugger?); is there a Win/Mac version that simulates 6510 ML along with all the memory map of the C-64? o good cross assemblers for 6510 (I've found CC65 and some others that look good) o any other tool that's useful in programming in assembly/ML, manipulating disk images, and debugging C64 files (preferably directly from D64/G64 or PRG files) I've found a treasure trove of C64 books online (bombjack.com) -- many that detail 6510 (Commodore-specific) programming including kernal calls, etc. A couple more questions: o What is the "best" (i.e., how do you do it) approach to disassembling a C64 file? For example, the Electronic Arts loader that's posted on this site - it's a very detailed disassembly and analysis. o Has anyone used ICU64 -- it's a program that runs in conjunction with VICE (and another emulator?) -- it shows memory locations, etc. in real time as the emulator runs programs. I didn't get it to load initially, but I'm sure it's an issue with .NET framework on my system. It looked really useful for taking apart code. o How can you tell (by looking at disassembler) which codes are instructions vs. data. Does the disassembler distinguish -- e.g., if it finds "A2 00", how does it know if it's "LDX $00" or just data? Thanks. |
 | Author: | Nate (guest: search) |
|---|---|---|
| Date: | Sun, Sep 05th, 2010 @ 14:06 ( . ) |
So many questions. :) I use cc65 and VICE, including its built-in monitor. It can disassemble and set breakpoints in drive RAM, and is thus very powerful for understanding protections. I usually set a read watchpoint on $255 or write wp on the command buffer at $200. That way you can see where the protection is loading data. I like Hexplorer as a hex editor. I don't use any more advanced tools to look at disks. I'm so used to the track format that editing the GCR in hex is easiest. ICU64 looks awesome but I haven't had time to try it out. Disassemblers just walk through memory linearly. It's up to you to tell it the range of data and instructions. |
| Author: | Kevin (guest: search) |
|---|---|---|
| Date: | Wed, Sep 08th, 2010 @ 05:05 ( . ) |
I didn't even know that VICE had a monitor in it. I'll have to check it out. |
| Author: | Kevin (guest: search) |
|---|---|---|
| Date: | Sun, Sep 12th, 2010 @ 13:45 ( . ) |
May be a little off-topic, but what about integrated development environments (IDEs) like these: [link] [link] [link] They look pretty useful, but they seem sort of generic in terms of being able to read .g64 and other Commodore disk formats. They seem more suited just for writing the code and compiling it into .prg files (which many of the command-line assemblers do). Anyone worked with these? |
 | Author: | SaxxonPike (registered user: 16 posts ) |
|---|---|---|
| Date: | Mon, Apr 04th, 2011 @ 11:07 ( . ) |
Thought about writing a disk image editor that worked on the bit level for D64 and G64 (maybe others if there's interest.) So far, I've written a utility that can bitshift entire tracks on a disk. It can also translate standard GCR both ways and manipulate directories and BAM for standard CBM DOS. I don't know if there are any fully featured utilities out there and this effort might be a bit redundant. But it's a personal exploration into the 1541 and its utilization of magnetic storage that I am mostly interested in. Someone might find it useful :) |
| Author: | Nate (guest: search) |
|---|---|---|
| Date: | Mon, Apr 04th, 2011 @ 16:05 ( . ) |
I'm working on the same thing now. It's going to be a small library for working with .nib/.g64 at the bit level. It will have convenience functions for shifting tracks, adjusting sync length, etc. The way it works is to dump the .nib/.g64 into a common .txt format. You can then edit it with any text editor. Then it can be parsed back into a .nib or .g64 It will not do d64, BAM, or other higher level functions as you can always do that yourself by interfacing with the txt file format. |
| Author: | Nate (guest: search) |
|---|---|---|
| Date: | Mon, Apr 04th, 2011 @ 19:50 ( . ) |
Ok, here's what I've gotten so far. The main question is how people want to work with the data. So please let me know what transformations are needed. The basic text format is: # NIB parsed file # Written on 2011/4/4 tracks 41 track 1 density 3 SYNC:41 157 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 14a 155 155 155 155 155 155 155 155 155 SYNC:33 149 14f 14e 14b 1cb 1cb 155 155 155 155 155 155 155 155 155 SYNC:41 ... The file lists the number of tracks and then entries for each track. Track 1 is in density 3. Then each SYNC (10 or more 1-bits) is listed, along with its bit-length. As you can see, nibread overestimated the standard DOS sectors here as 41 bits instead of 40. Data is in 10-bit hex format, big-endian. So 10 bits of sync is 3ff. For sync-less tracks, the data is just output as raw 10-bit GCR symbols (no "SYNC" listed in it but otherwise the same as track data). Operations you can do already: * Insert, change, delete GCR symbols * Change SYNC length, delete SYNCs, add them, etc. Adding soon to the code: * Shift track: output 7 .nib files with shifts of 1 to 7 bits for a given track * Density conversion: given a range of bits within a track, convert the raw data from one density to another Any other suggestions? Like or hate the data format? |
 | Author: | Lord Crass (guest: search) |
|---|---|---|
| Date: | Mon, Apr 04th, 2011 @ 21:08 ( . ) |
Looks great! How about an option to display the data as 8-bit GCR as well? I think most folks who are used to editing raw GCR are probably more familiar with that format. |
| Author: | Nate (guest: search) |
|---|---|---|
| Date: | Tue, Apr 05th, 2011 @ 02:45 ( . ) |
Sure, will do that. I assume you mean a set of 5 bytes (40 GCR bits). I will also add a comment with the 4-byte decoded version alongside so it looks like this: aa bb cc dd ee ff aa bb cc dd # 11 22 33 44 55 66 77 88 The decoded version would not be parsed when rewriting the data. Of course, you can rework the GCR lines themselves any way you want (any number of bytes per line, different line lengths, etc.) |
 | Author: | hyper active (registered user: 296 posts ) |
|---|---|---|
| Date: | Thu, Apr 07th, 2011 @ 06:22 ( . ) |
what would be really good is if we could take all the raw tracks and splice them back together in a g64 or .nib file. |
| Author: | Nate (guest: search) |
|---|---|---|
| Date: | Thu, Apr 07th, 2011 @ 16:04 ( . ) |
Of course... you extract all the tracks into a simple text file, edit as you wish, then re-run it to generate the corresponding .nib/.g64 file. I'll put this online soon. It needs more cleanups. |
 | Author: | Pete Rittwage (registered user: 558 posts ) |
|---|---|---|
| Date: | Fri, Apr 08th, 2011 @ 08:48 ( . ) |
This is an interesting and easy way to accomplish this- I like it. I'm not so sure about the 10-bit HEX, though- I've never worked in that format before. Is this common in modern embedded development or something? |
| Author: | hyper active (registered user: 296 posts ) |
|---|---|---|
| Date: | Fri, Apr 08th, 2011 @ 19:55 ( . ) |
hmm. this is not quite what I had in mind. I've been using the hidden raw track dump mode and each track has been extracted into a file, tr1.0d3, tr2.0d3, tr3.0d3 etc. When Lord Crass showed me what to change on track 12 recently, I simply loaded up the tr12.0d3 file in ultra edit. The problem is that I have no way to splice these tracks together in to nib or g64, and that was what I was hoping that Nate's program could do. |
| Author: | Nate (guest: search) |
|---|---|---|
| Date: | Sun, Apr 10th, 2011 @ 13:34 ( . ) |
Made some more progress on this and now have some questions. The code currently lets you do the following by editing the text dump of an image: * Read/write .nib or .g64 * Insert, change, delete GCR bytes * Change SYNC length, delete SYNCs, add them, etc. * Format is 5:4 bytes (GCR to decoded), two sets of these per line (10 bytes GCR) There are also commands to do the following: * Shift track: output 7 .nib or .g64 files with rotates of 1 to 7 bits for a given track # I'm looking at adding support for these features but uncertain which ones are most useful: * Align a given track (rotate in image), based on the following: - First sync found (10 bits of 1) - Sector number (does some decoding) - GCR pattern Are there any others that would be useful? Start track at longest sync? Of course, you can implement arbitrary track-to-track alignments with this by rotating track 1 to start at sector 0, track 2 to sector 15, or whatever. * Density conversion -- How should this be done? I understand that if you want to convert a range of data from higer density to lower, you just discard bits. But I think there's no way to go from lower to higher. For density, I support the g64 per-byte extended map so you can set arbitrary density for byte ranges within each track. Of course, you lose that info if you write out a .nib. Comments welcome. |
| Author: | Pete Rittwage (registered user: 558 posts ) |
|---|---|---|
| Date: | Sun, Apr 10th, 2011 @ 21:00 ( . ) |
There are already some alignments done in the code (used in nibconv and nibwrite, contained in prot.c). There is a V-MAX, and there *was* a Rapidlok one, but it seems to be lost/gone. I think someone did a better one and may submit it, though. The PirateSlayer alignment used to shift the track for you also when converting, and it looked for the magic sequence automatically. I'll look and see why that is gone now, too. |
| Author: | Nate (guest: search) |
|---|---|---|
| Date: | Tue, Apr 12th, 2011 @ 22:11 ( . ) |
Any answers to my other questions on density conversion? Does anything support the extended g64 density map such as nibwrite or VICE? |
| Author: | Pete Rittwage (registered user: 558 posts ) |
|---|---|---|
| Date: | Tue, Apr 12th, 2011 @ 22:58 ( . ) |
I don't believe anything supports alternate densities within a track. VICE knows nothing of density, I am sure of that. It only sees the whole bytes in the G64 no matter what. |
| Author: | Lord Crass (guest: search) |
|---|---|---|
| Date: | Wed, Apr 13th, 2011 @ 11:00 ( . ) |
Is there any value in adding a density conversion option? Usually if data was read from a real 1541 at the wrong density, it's not very useful since you can no longer guarantee that it conforms to the C= GCR standard. It's possible to have too many 0's come in and clock in a phantom bit, or perhaps read in 10 1's in a row. This would probably be rare, but I think the chance of data corruption is high enough that it's not worth attempting to fix through software. For the rare disk that does use multiple densities on the same track, it's probably easier to just read it at each density and paste it together after, marking those bytes with the appropriate speed zone, should any program decide to use that data in the future. Or were you thinking of using this tool for other disk images as well, where the drive used to read the data doesn't have faulty shift registers or issues with reading sync? In that case, it might be useful. |
| Author: | Nate (guest: search) |
|---|---|---|
| Date: | Thu, Apr 14th, 2011 @ 17:29 ( . ) |
That's what I was wondering -- how you guys determined which data was needed when the density used to read the image was wrong. I think you are just disassembling the protection code and looking for the pattern it is searching for and what it sets the density to before doing so, right? I'll make it easy to specify density for ranges of bytes in a track. This will get written out to the g64 correctly (using the extended speed map) but will be lost if writing a .nib file. This way you can read out the track multiple times with nibread and then piece back together each track by just putting in the text file something like: (2, xx xx xx) (3, yy yy) BTW, does nibwrite or VICE support the g64 extended speed map? It doesn't seem like it. |
| Author: | Lord Crass (guest: search) |
|---|---|---|
| Date: | Thu, Apr 14th, 2011 @ 19:56 ( . ) |
Yes, I just disassembled the drive code to see what it was doing and how that data was used on the C64 side. As far as I know, Vice doesn't use the speed zone data. I don't think nibwrite does either. Based on the tight timing loops when writing out a track, I'm not sure if there's enough cycles available to check the speed zone and adjust the density before writing the next byte. There might be a missed byte if density needs to change on the fly. Pete would know more about this. |
| Author: | Pete Rittwage (registered user: 558 posts ) |
|---|---|---|
| Date: | Thu, Apr 14th, 2011 @ 20:25 ( . ) |
Sure, it might be able to be done with a small change in the writing code. At least on the 1571@2MHz, it would be easy. The 1MHz code is kind of tight. I'm not sure it's worth the trouble for this one protection that is used on a couple disks, though, but for the sake of completeness... I'm not sure how many cycles it takes to change density, so it would need experimentation. Another idea is to do something where we wait for a signature to pass by and then write, which you could use to append data to a track at a different density. |
 | Author: | J Achernar (registered user: 36 posts ) |
|---|---|---|
| Date: | Fri, Apr 15th, 2011 @ 23:10 ( . ) |
From working with Black Hole, neither VICE or CCS64 uses the speed zone data, but it does not hurt anything either. I manually added the speed zone data to Black Hole, even though it was not necessary to make it work in the emulators, so that it was as accurate as I could make it. |
| Author: | Nate (guest: search) |
|---|---|---|
| Date: | Fri, May 27th, 2011 @ 21:42 ( . ) |
I'm still working on this. Had a long break due to work but might get some time this weekend, between prepping house for construction. I've got g64/nib parsing and conversion and some various GCR formatting features. I need to implement the rest of the tool, then I'll notify you. |
| Author: | hyper active (registered user: 296 posts ) |
|---|---|---|
| Date: | Mon, Jun 20th, 2011 @ 13:35 ( . ) |
Hiya. Will your program work on nb2 files? Nibread allows you to dump a track or tracks multiple times and read it out in several different densities. Thanks. |
| Author: | Nate (guest: search) |
|---|---|---|
| Date: | Tue, Jun 21st, 2011 @ 00:35 ( . ) |
On 06/20/2011 @ 13:35, hyper active wrote :
Hiya. : Will your program work on nb2 files? : Nibread allows you to dump a track or tracks multiple times and read it out in several different densities. : Thanks. -- I've been busy and haven't finished this tool yet. Not yet but I'll consider supporting them. You can always work with g64s pieced together from each density track. BTW, I've always thought nibread should read data at the fastest bitrate and then downsample in nibconv. Is there any problem with this? That way any of the slower bitrate density settings could be recreated in software and tracks wouldn't have to be re-read. |
| Author: | Pete Rittwage (registered user: 558 posts ) |
|---|---|---|
| Date: | Tue, Jun 21st, 2011 @ 09:37 ( . ) |
I recall trying that at some point, but the problem is that you often lose framing when reading at the wrong bit rate, and it can't be put back together again reliably. Timing isn't perfect on belt driven drives and at unknown rotation speed, and I don't think you can test rotation speed without writing data. Even worse, some drives (some 1541-II's especially) will return random data that isn't based on the actual flux stream, when out of framing. I can look at it again if there's something I missed, though. |
| Author: | Nate (guest: search) |
|---|---|---|
| Date: | Wed, Jun 22nd, 2011 @ 13:14 ( . ) |
Yeah, I understand that if you have a low density of 1 bits, choosing a bitrate setting that is too high would result in spurious 1's being clocked into the shift register. Basically, you'd be creating "weak bits" if reading a low bitrate source at a higher rate. |
| Author: | Pete Rittwage (registered user: 558 posts ) |
|---|---|---|
| Date: | Sat, Jul 02nd, 2011 @ 15:02 ( . ) |
I can try reading all at the lowest density and calculating. Do you have a routine to recalculate at another bit rate that I can experiment with? |
| Author: | Pete Rittwage (registered user: 558 posts ) |
|---|---|---|
| Date: | Sat, Jul 02nd, 2011 @ 15:02 ( . ) |
Pseudo-code or an explanation would be fine... |
 | Author: | Fungus (registered user: 20 posts ) |
|---|---|---|
| Date: | Sun, Jul 03rd, 2011 @ 07:50 ( . ) |
I don't think 1571's and 1541-II's should be used on some disks to dump with as they don't function exactly the same as an older 1541. Been researching some of the hardware differences, so that vice disk emulation can be re-done lately. I've found that II's and 71's can not read with byte_sync disabled, it simply does not work at all on those drives. That's the reason some games were released again with changes to the protection so that they would function on those drives. Star Rank Boxing is an excellent example of this, due to the way it checks the syncs and data before and after them. Lord Crass can explain how these checks work in greater detail than I can at the moment. =] Reading disks with multiple densities on a track or abnormal densities is just going to be a problem on any drive, the protections which do such stuff need to be analyzed and custom dumping would have to be done for such disks as you know it's a bit hard to auto detect the density reliably. Also, very much looking forward to the release of G64/NIB editor. I can't wait ;) |
| Author: | hyper active (registered user: 296 posts ) |
|---|---|---|
| Date: | Wed, Jul 13th, 2011 @ 00:46 ( . ) |
si. Huri! Huri! Huri! |
| Author: | Pete Rittwage (registered user: 558 posts ) |
|---|---|---|
| Date: | Thu, Jul 14th, 2011 @ 19:22 ( . ) |
Hi Nate, and all, (I am cross-posting this to the Nate's XUM-1541 devel list also to see if there is any interest in my helping to implement something like this on another level.) Thanks very much for taking your valuable time for old hobbies. I am sorry for my brain-dump format of this message, as it will be long and rambling. The 'nibedit' idea reminds me of something I've been thinking about for a while with regard to NIB files as Markus created, and all the different formats of emulator files we have to deal with (not to mention X cables). It seems to me that the image formats as passed down over the years are inherently inflexible. I've considered getting rid of the NIB format altogether and converting to a more modern XML-tagged "description" of the disk data that is in the file as we captured it. These files could contain lots of different tags to describe everything down to whatever level we have access to and at any level of detail. For example, a NIB image as read now into an XML format would contain tags with the timestamp, OS, program version, interface/cable type, interface firmware revision, drive type, drive speed, drive ROM hash, switches used, IHS or no, SRQ or parallel, halftracks, publisher, copyright, etc. (down to incredible detail). Some of this is currently in an accompanying "log" file that most people unfortunately don't submit to me. Track or sector tags would contain: *) detected density *) detected track types (nosync/killer) *) length as measured. *) description of following data chunk A data chunk could be: *) a "NIB" revolution, which is 1.x disk revolutions of GCR as read from the shifter (raw GCR) *) a "track cycle" of data from IHS pulse to IHS pulse (hole start or hole end?) *) fully decoded GCR data (D64-type data) *) separate raw sector headers and raw data sectors (breakout) *) HEX-formatted values of GCR data *) a single non-standard, or sync-to-sync GCR run *) description of "signature track". (So replace Pirateslayer track data with <tr>2<sig>pslayer1</sig></tr> *) flux transition timings, used with different imaging hardware *) compression type of any of the above The benefits are endless, and it could be used to convert to existing format, so it's backwards-compatible. You could "patch" the images using tags which we would could just be a another tag later in the XML file that supercedes the other tag. So edit a track, sector, or just a pattern of bytes using a description. This could be used for high score tracks/sectors, protection descriptions or patches, image differentials (alternates), or simple editing. If you want to "edit" a track, just define a new track at the end of the file with the new data. This could really be an "open" emulator media format like we've never had before. For any media for any emulator type. -Pete |
| Author: | hyper active (registered user: 296 posts ) |
|---|---|---|
| Date: | Sat, Jul 30th, 2011 @ 18:05 ( . ) |
Hi Nate. Any idea when you'll be releasing your utility to edit nib and g64 files? Or have you decided to leave it and let Peter develop his .xml system? Thanks. |
| Author: | Nate (guest: search) |
|---|---|---|
| Date: | Sun, Jul 31st, 2011 @ 15:38 ( . ) |
It will be a while. First in line for hobby time is to get the firmware update utility for ZF written so we can do another release of OpenCBM for it. I'm also helping get SRQ nibbling working on the 1571. So it will be a while until I can finish and release this tool. There has been additional discussion of Pete's ideas, and I'm not getting involved in that so he can do whatever he wants. |
| Author: | Pete Rittwage (registered user: 558 posts ) |
|---|---|---|
| Date: | Sun, Jul 31st, 2011 @ 15:43 ( . ) |
I was not trying to subvert Nate's tool, and do not plan in the immediate future to change anything. I currently do not have the time to implement a new file format. A GUI-based G64 editor would be the way to go for this, but someone with more time would need to develop it. |
| Author: | hyper active (registered user: 296 posts ) |
|---|---|---|
| Date: | Thu, Sep 01st, 2011 @ 18:45 ( . ) |
Any idea on when you might release the utility? what sort of time frame are we looking at, October, Christmas? new year? Thanks, not trying to sound impatient or anything. |
| Author: | Nate (guest: search) |
|---|---|---|
| Date: | Tue, Sep 13th, 2011 @ 17:40 ( . ) |
It's last in line, behind getting the next ZoomFloppy release out. That's still got a bit of work remaining too, but the end is in sight. |
| Author: | hyper active (registered user: 296 posts ) |
|---|---|---|
| Date: | Thu, Dec 01st, 2011 @ 02:06 ( . ) |
will your editor be released in time for Christmas? |
| Author: | hyper active (registered user: 296 posts ) |
|---|---|---|
| Date: | Thu, May 31st, 2012 @ 06:39 ( . ) |
Hi nate. any ideas about the release date of your nib/g64 editing tool? Cheers. |