Jekyll2023-07-25T21:12:00+00:00https://dellagustin.github.io/feed.xmlDellagustin’s personal blogThis is a personal blog about any random topic that I want to write about!Saying ‘Hello World’ in the MSX-DOS Operating System2023-07-25T00:00:00+00:002023-07-25T00:00:00+00:00https://dellagustin.github.io/retro-computers/2023/07/25/hello-world-on-mxs-dos<p>I recently published a blog post on <a href="./2023-06-09-hello-world-on-dos-1.md">Saying ‘Hello World’ in the IBM-PC with MS-DOS 1.0</a>. After that, I got curious about doing the same on MSX-DOS.</p>
<p>MSX-DOS was a port of MS-DOS/PC-DOS for the MSX computers on 1984. The MSX was not a single computer from a specific company, but rather a standardized architecture for 8-bit machines established through a partnership of Microsoft and ASCII Corporation in Japan.</p>
<p>The first computer based on this standard was the Mitsubishi ML-8000, released in 1983.</p>
<h2 id="saying-hello">Saying Hello</h2>
<p>This time I’ll skip a lengthier architecture talk, as I already touched that in my previous post. What we need to know:</p>
<ul>
<li>The MSX computers have a Zilog Z80 (or compatible) as its CPU under the hood</li>
<li>Like in MS-DOS, there was an underlying API for basic stuff, like showing a string</li>
<li>Like in MS-DOS, “external commands” are <code class="language-plaintext highlighter-rouge">.COM</code> files with pure machine code that are loaded on the memory address <code class="language-plaintext highlighter-rouge">0x100</code> and executed from there</li>
<li>Unlike MS-DOS, the MSX-DOS API does not use software interrupts, but rather a simple subroutine invoked with the <code class="language-plaintext highlighter-rouge">CALL</code> instruction, moving the execution to the area in RAM where the Operational System is loaded</li>
</ul>
<p>The Z80 CPU was used in many other computers of this era and also arcade games, such as the iconic Pac-Man.</p>
<p>Now, how do we create a <code class="language-plaintext highlighter-rouge">HELLO.COM</code> command using the MSX-DOS API to print a <em>Hello World</em> message in the screen Well, MS-DOS had the handy <code class="language-plaintext highlighter-rouge">DEBUG</code> application, which allowed us to write machine code to memory, executed it, unassemble it, inspect it and save it to disc, but that’s not present in MSX-DOS (even though a good soul wrote a port for it, see <a href="https://gitlab.com/Emily82/debugger-debug.com-for-msx">https://gitlab.com/Emily82/debugger-debug.com-for-msx</a>).</p>
<p>So, reading the <a href="https://konamiman.github.io/MSX2-Technical-Handbook/md/Chapter3.html">MSX2 Technical Handbook, Chapter 3 - MSX-DOS</a>, Section 2.4 - <em>External Commands</em>, I got an interesting surprise. It suggests the following BASIC program for writing bytes to a file:</p>
<pre><code class="language-basic">100 '***** This program makes "CLS.COM" *****
110 '
120 OPEN "CLS.COM" FOR OUTPUT AS #1
130 '
140 FOR I=1 TO 8
150 READ D$
160 PRINT #1,CHR$(VAL("&H"+D$));
170 NEXT
180 '
190 DATA 1E,0C,0E,02,CD,05,00,C9
</code></pre>
<p>This is the very simple external command <code class="language-plaintext highlighter-rouge">CLS.COM</code>, written byte-by-byte to the disk.</p>
<p>Isn’t that mind blowing? Writing a basic program to create a MSX-DOS program. There are of course more sophisticated ways of doing that, but I wanted to do it with the tooling I had just with the computer plus the operating system, no extra tooling.</p>
<p>We can adapt that to create our own <code class="language-plaintext highlighter-rouge">HELLO.COM</code> command, all we need to do is to figure out the bytes that we need for the instructions that will call the MSX-DOS API to print our message in the screen, and also the message itself.</p>
<p>Let’s look into the API then, referring again to the Technical Handbook, now section 4 of Chapter 3, <em>System Call Usage</em>.</p>
<p>It says that we need to enter the API function number in the C register of the CPU, and then call the sub-routine at address <code class="language-plaintext highlighter-rouge">0x0005</code>. The function number to send a string to the screen is <code class="language-plaintext highlighter-rouge">0x09</code>, and it takes a single argument, the memory address where the string is located. Just like in MS-DOS, the character <code class="language-plaintext highlighter-rouge">$</code> indicates the end of the string.</p>
<p>Let’s write the code to do that in assembly, just as a reference, because we will need to translate it into machine language:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>LD C,09H
LD DE,0109H
CALL 0005H
RET
</code></pre></div></div>
<p>Note that the second instruction is setting the register DE to <code class="language-plaintext highlighter-rouge">0x109</code>, this is the address where the string will be. I’m already setting it here, but it is only because I already translated this to machine code, and I know exactly the memory address that follows the <code class="language-plaintext highlighter-rouge">RET</code> instruction. Keep in mind, this code will be loaded into the memory address <code class="language-plaintext highlighter-rouge">0x100</code>.</p>
<p>Ok, now let’s translate that assembly code to machine code, using this convenient <a href="https://clrhome.org/table">Z80 opcode table</a>. The result is (memory address and opcode in hex):</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>M OP ASM
100 0E 09 LD C,09H
102 11 09 01 LD DE,0109H
105 CD 05 00 CALL 0005H
108 C9 RET
</code></pre></div></div>
<p>So, the code itself is the following sequence of bytes: 0E, 09, 11, 09, 01, CD, 05, 00, C9</p>
<p>We still need to append our text message to that (<code class="language-plaintext highlighter-rouge">Hello World!$</code>). To convert that to hex, I’ll use the list of MSX characters and control codes at <a href="https://www.msx.org/wiki/MSX_Characters_and_Control_Codes">https://www.msx.org/wiki/MSX_Characters_and_Control_Codes</a>.</p>
<p>The result is: 48, 65, 6c, 6c, 6f, 20, 57, 6f, 72, 6c, 64, 21, 24</p>
<p>Now, all we need to do is adapt the basic program:</p>
<pre><code class="language-basic">100 '***** This program makes "HELLO.COM" *****
110 '
120 OPEN "HELLO.COM" FOR OUTPUT AS #1
130 '
140 FOR I=1 TO 22
150 READ D$
160 PRINT #1,CHR$(VAL("&H"+D$));
170 NEXT
180 '
190 DATA 0E,09,11,09,01,CD,05,00,C9,48,65,6C,6C,6F,20,57,6F,72,6C,64,21,24
200 CLOSE
</code></pre>
<p>Note that I added a <code class="language-plaintext highlighter-rouge">CLOSE</code> statement. It was missing on the example from the manual. Without it, the content was not actually written to the file!</p>
<p>At this point it is good to mention that you can emulate an MSX on the web at <a href="https://webmsx.org/">https://webmsx.org/</a>.</p>
<p>All we need to do is type the program above in the basic interpreter and run it, it will create the file <code class="language-plaintext highlighter-rouge">HELLO.COM</code>.</p>
<p>Then we go back to the MSX-DOS and execute it, and the magic happens! You will be greeted with a <em>Hello World!</em> message!</p>
<p>In my setup, I loaded the MSX-DOS disk to disk 1 (A), and created the BASIC program and HELLO.COM program on disk 2 (B).</p>
<p>Here are some screenshots:</p>
<p><img src="/assets/retro-computers/WebMSX-DOS-hello-basic.png" alt="Basic code for writing a hello world command in MSX-DOS, webmsx emulator" /></p>
<p><img src="/assets/retro-computers/WebMSX-DOS-hello-cmd.png" alt="Hello world command running in MSX-DOS, webmsx emulator" /></p>
<h2 id="closing-words">Closing words</h2>
<p>I was quite surprised with the fact that I ended up writing a simple BASIC program to create the external command for me, but at the end this was a fun experience.</p>
<p>The MSX was an interesting spec, but arriving two years later than the IBM-PC, it did not have a big chance of becoming THE standard for home computers (which IBM did without even wanting to do so).</p>
<p>These “hello world”s are an interesting way to start grasping how these machines work on a lower level. I hope you like it and try it for yourself!</p>
<h2 id="references">References</h2>
<ul>
<li>https://en.wikipedia.org/wiki/MSX</li>
<li>https://en.wikipedia.org/wiki/MSX-DOS</li>
<li>https://konamiman.github.io/MSX2-Technical-Handbook/</li>
</ul>I recently published a blog post on Saying ‘Hello World’ in the IBM-PC with MS-DOS 1.0. After that, I got curious about doing the same on MSX-DOS.Free/Libre and Open Podcasting - a conversation starter2023-07-13T00:00:00+00:002023-07-13T00:00:00+00:00https://dellagustin.github.io/podcasting/2023/07/13/free-libre-open-podcasting<p>I recently posted the following on my <em>Mastodon</em> account for open source:</p>
<blockquote>
<p>Random thought, we should have a term for open podcasting (even though @adam@podcastindex.social claims that just “podcast” already means that :)) - I’m thinking about Free/Libre Open Podcasting, the acronym is terrible though, FLOP. It of course borrows from #FLOSS.</p>
</blockquote>
<p>The post: <a href="https://fosstodon.org/@dellagustin/110706347484988427">https://fosstodon.org/@dellagustin/110706347484988427</a></p>
<p>It got a pair of replies (one private) that indicated I should elaborate, and I think a blog post is more suitable then a microblog thread.</p>
<h2 id="what-do-i-mean-by-freelibre-and-open-podcasting">What do I mean by Free/Libre and Open Podcasting?</h2>
<p>By <em>Free/Libre and Open Podcasting</em> (or FLOP, a terrific acronym 😀), I mean a concept and corresponding movement that establishes and defends the freedoms of podcasters and listeners. It is analogous the <em>Free/Libre and Open Source Software</em> (FLOSS), in the way it defines and defends the freedoms of software users. If you are wondering about the need for <em>Libre</em> in the term, check the Wikipedia article <a href="https://en.wikipedia.org/wiki/Alternative_terms_for_free_software">Alternative terms for free software</a> with a special attention to the ambiguity of the English word <em>free</em>.</p>
<p>I’m imagining freedoms such as:</p>
<ul>
<li>Listeners are free to listen to, interact with and support their podcasts in their platform and software of choice</li>
<li>Podcasters are free to host their podcasts and interact with their audience and/or community in their platform of choice</li>
</ul>
<p>These freedoms are supported by open standards (such as RSS 2.0 and the <a href="https://github.com/Podcastindex-org/podcast-namespace">podcast namespace</a>), which promote interoperability, but not necessarily by open source software (although that would be good secondary goal).</p>
<h2 id="why-such-a-concept-and-movement-is-important">Why such a concept and movement is important?</h2>
<p>The article <a href="https://podstandards.org/2022/03/24/podcasting-is-better-when-its-open/">What is open podcasting, and why does it matter?</a> at podstandards.org already does a good job explaining this, but, I’d like to put it in my own words.</p>
<p>A small digression first, just as the term <em>open source software</em> is not sufficient with respect to the freedoms of software users (therefore FLOSS), the term <em>open podcasting</em> is not sufficient, in my opinion, to represent the fight for the freedoms of podcast listeners and creators.</p>
<p>Back to the importance topic.<br />
Since a long time ago, Apple had THE podcast index, iTunes. It was used by many podcast clients as its main source of content for searching and later subscribing to podcasts, and it still is, to a large degree (although, thankfully, we now have alternatives). Even so, it was open to external consumers (i.e. other players than Apple Podcasts) and it had little policing.</p>
<p>In recent years though, other large corporations made, or are in the process of making moves to close podcasting. Namely, Spotify has coopted podcasts into becoming own exclusive shows (I’ll not call them podcasts, if they are not free/libre and open), and most recently, Google is adding podcasts to YouTube and showing signs it wants to have a lot of control over them.</p>
<p>Although having your podcast in Spotify and YouTube can bring your listeners and grow your podcast, you also risk building your own prison, specially if you end up depending on their monetizations platforms. We have seen arbitrary changes in monetization schemes drastically affecting content creators in the past, e.g. with YouTube and Twitch.</p>
<p>Besides that, I have heard many podcast creators commenting on adjustments they had to do to their podcast to remain in Spotify, including the podcast I’m part of.</p>
<h2 id="beyond-listening">Beyond listening</h2>
<p>I added the words <em>interact</em> and <em>support</em> to the first freedom above purposely. It means that the freedom and openness should go beyond the listening experience and I’m thinking specifically about two points, social media and monetization (although, it should not be restricted to that in any way).</p>
<p>We have seen the Twitter takeover by a single autocract billionaire, and the platform seems to be melting by the day. With that there was some movement towards freedom and openness with a timid migration into the <em>Fediverse</em>, but more and more I see content creators jumping from one non-free/libre platform to another, e.g. Bluesky and Threads, again having to rebuild their network and locking themselves without any chance of sovereignty.</p>
<p>In a similar way, podcasters tend to use proprietary monetization platforms, such as Patreon and PayPay, where they can get “deplatformed” in an arbitrary way. Here, free/libre and open alternatives, specially not related to cryptocurrencies, which is a divisive topic, are lacking.</p>
<p>What I mean here, is that the movement should fight for freedom and openness to everything that is required for a good and sustainable experience for podcast listeners and creators, thus creating, contributing to and promoting open standards in all these fronts and also promoting the usage of the tools that support them.</p>
<h2 id="closing-words">Closing words</h2>
<p>I hope this clarifies the idea for <em>Free/Libre and Open Podcasting</em> and that it also serves the purpose of fueling conversations nad interactions that will move this forward.</p>
<p>If you are seeing a contradiction between preaching for freedom and openness while having a proprietary comments platform (at the moment of writing, Disqus), please know I’m not proud of that. As a free and open alternative, if you have comments, please reply the the Mastodon post that originated this post: <a href="https://fosstodon.org/@dellagustin/110706347484988427">https://fosstodon.org/@dellagustin/110706347484988427</a></p>
<h2 id="links-and-references">Links and References</h2>
<ul>
<li><a href="https://www.gnu.org/philosophy/floss-and-foss.en.html">https://www.gnu.org/philosophy/floss-and-foss.en.html</a></li>
<li><a href="https://www.gnu.org/philosophy/free-sw.html">https://www.gnu.org/philosophy/free-sw.html</a></li>
<li><a href="https://en.wikipedia.org/wiki/Alternative_terms_for_free_software">https://en.wikipedia.org/wiki/Alternative_terms_for_free_software</a></li>
<li><a href="https://podstandards.org/2022/03/24/podcasting-is-better-when-its-open/">https://podstandards.org/2022/03/24/podcasting-is-better-when-its-open/</a></li>
<li><a href="https://en.wikipedia.org/wiki/Fediverse">https://en.wikipedia.org/wiki/Fediverse</a></li>
</ul>I recently posted the following on my Mastodon account for open source:Saying ‘Hello World’ in the PC-DOS 1.0 Operating System2023-06-26T00:00:00+00:002023-06-26T00:00:00+00:00https://dellagustin.github.io/retro-computers/2023/06/26/hello-world-on-dos-1<p>I like spending some of my time learning more about the architecture of retro computers. That’s an interest that comes from a very early love for computers in my life and that includes video games. From time to time I read about this topics and some time ago I was interested in the IBM PC (IBM 5150), launched in 1981.</p>
<p>That’s a very important device in the history modern computers, as it gave origin to the x86 architecture and also the MS-DOS/PC-DOS (Disk Operating System) family of operating systems. Note that DOS is a term that can refer to other operating systems of the time, but throughout this post, DOS will refer to MS-DOS/PC-DOS.</p>
<p>After seeing a video of someone writing assembly code on DOS using the <code class="language-plaintext highlighter-rouge">DEBUG.COM</code> application, I thought it would be a nice little exercise, really, just a classic <em>“Hello world”</em>, but in a very old school way.</p>
<p>In this post I’ll talk briefly about the architecture of these early machines and show you how to write a <em>Hello world</em> in machine code for the IBM PC in DOS 1.0.</p>
<h2 id="the-ibm-pc-architecture-in-a-nutshell">The IBM PC architecture in a nutshell</h2>
<p>Understanding the system architecture in depth is not really needed to write a <em>“Hello World”</em>, but the whole point of writing it in assembly (actually direct machine code) is to do it as close as possible to the hardware and experience some of the nitty gritty of these architectures.</p>
<p>In general the architecture of 8-bit microcomputers that era would look like in the diagram below:</p>
<p><img src="/assets/retro-computers/general-8-bit-architecture.svg" alt="General 8-bit microcomputer architecture" /></p>
<p>Specific to our scenario:</p>
<ul>
<li>the CPU is an Intel 8088,</li>
<li>the RAM ranges form 16 to 640 KB,</li>
<li>the Video will be a Monochrome Display Adapter (MDA), although this does not specifically affect our task, and</li>
<li>the Read and Write Storage will be a 360 KB floppy disk, from where the operating system is load, and where we will store our program.</li>
</ul>
<p>Looking into the software architecture, this is how it looks like:</p>
<p><img src="/assets/retro-computers/hardware-and-software-layers.drawio.svg" alt="Hardware and software layers" /></p>
<p>At start up, the CPU will fetch instructions directly from the BIOS.
The BIOS, Basic Input and Output System, will load the Operating System (OS) and provide an abstraction layer for it so that it can interface with the hardware. The OS, in turn, will provide some higher level features directly to the user, like interacting with file systems and loading and running programs, as well as providing some basic reusable routines and serving as yet another abstraction layer for the programs it loads and runs.</p>
<h2 id="now-to-the-action">Now, to the action</h2>
<p>I don’t have IBM PC or DOS 1.0 disk at hand, and I assume you don’t have one either.</p>
<p>Luckily, it is very easy to emulate this device today. I’ll be using the excellent site PC pcjs.org (<a href="https://www.pcjs.org/machines/pcx86/ibm/5150/mda/">here</a> is a direct link to the IBM PC emulator), which emulates devices directly in the browser.</p>
<p>PCem is also a good option to emulate it on your PC.</p>
<p>Before we go on, let’s check what our program is going to do.
We need to do basically two things:</p>
<ol>
<li>Tell the operating system to show a message on the screen,</li>
<li>Tell the operating system we are done.</li>
</ol>
<p>We are going to do that with the DOS API. To consume this API we need to invoke software interrupts with the <code class="language-plaintext highlighter-rouge">INT</code> instruction. That will divert the flow of the code being executed by the processor to code that belongs to the operating system, which will bring the flow back to our program once it is ready (or not, if we are done).</p>
<p>The main DOS API uses interrupt vector <code class="language-plaintext highlighter-rouge">21h</code>, and the function to display the message is selected by setting the register <code class="language-plaintext highlighter-rouge">AH</code> to <code class="language-plaintext highlighter-rouge">09h</code> and the <code class="language-plaintext highlighter-rouge">DX</code> register to the memory address where the message is located. This function considers that messages end in the <code class="language-plaintext highlighter-rouge">$</code> character. To end the program, we call the interrupt after setting register <code class="language-plaintext highlighter-rouge">AH</code> to <code class="language-plaintext highlighter-rouge">00h</code>.</p>
<p>Now, some nitty gritty about the <code class="language-plaintext highlighter-rouge">DEBUG</code> version in DOS 1.0:</p>
<ul>
<li>it does not assemble (unlike the avengers), so we need to “code” using machine code directly - later versions can create machine code from assembly commands,</li>
<li>it does not save your program to disk, unless the file already exists.</li>
</ul>
<p>So, first, lets create the empty file <code class="language-plaintext highlighter-rouge">HELLO.COM</code> with the following command:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>copy con hello.com
</code></pre></div></div>
<p>then press <em>CTRL+Z</em> (you will see <code class="language-plaintext highlighter-rouge">^Z</code> on the screen) and press <em>ENTER</em>, you will see the message <code class="language-plaintext highlighter-rouge">1 File(s) copied</code>.</p>
<p>Let’s get to the fun part. We’ll start using <code class="language-plaintext highlighter-rouge">DEBUG</code>, type the following command:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>debug hello.com
</code></pre></div></div>
<p>You will be greeted by a simple <code class="language-plaintext highlighter-rouge">-</code> followed by a blinking cursor.</p>
<p>We will add the following instructions in machine code, starting from the memory address <code class="language-plaintext highlighter-rouge">100h</code>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Memory address (in hex) Instruction Machine code (in hex)
100 MOV AH,09 B4 09
102 MOV DX,010B BA 0B 01
105 INT 21 CD 21
107 MOV AH,00 B4 00
109 INT 21 CD 21
</code></pre></div></div>
<p>I have converted those instructions to machine code manually using this opcodes list: <a href="https://www.pastraiser.com/cpu/i8088/i8088_opcodes.html">https://www.pastraiser.com/cpu/i8088/i8088_opcodes.html</a></p>
<p>Notice that on memory address <code class="language-plaintext highlighter-rouge">102h</code> we are setting the register <code class="language-plaintext highlighter-rouge">DX</code> to <code class="language-plaintext highlighter-rouge">10Bh</code>, it is exactly the address that comes after our last instruction, we will place our message there later.</p>
<p>To enter those instructions, use the following command (remember we are still inside <code class="language-plaintext highlighter-rouge">DEBUG</code>):</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>e 100 B4 09 BA 0B 01 CD 21 B4 00 CD 21
</code></pre></div></div>
<p>Now check if you did it correctly with the <code class="language-plaintext highlighter-rouge">u 100</code> (unassemble) command. <code class="language-plaintext highlighter-rouge">DEBUG</code> will print the instructions and you can compare it with the ones above.</p>
<p>Great! Now, let’s enter the message! use the command below:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>e 10B "Hello world!$"
</code></pre></div></div>
<p>All right, we are ready to test! Use the following command:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>g=100
</code></pre></div></div>
<p>With this, debug will run the code you have written, starting on the memory address <code class="language-plaintext highlighter-rouge">100h</code>, you should see the message <em>Hello world! Program terminated normally</em>.</p>
<p>All there is left to do now is to save our program to disk. For that, we need to write the program size to the register <code class="language-plaintext highlighter-rouge">CX</code>. Use the command <code class="language-plaintext highlighter-rouge">RCX</code> and enter the value <code class="language-plaintext highlighter-rouge">18</code> (it is the size of our program in bytes, 24, converted to hex), then use the command <code class="language-plaintext highlighter-rouge">w</code>. This will write your program to disk. Now quit debug with the command <code class="language-plaintext highlighter-rouge">q</code> and run your program by typing its name in the command prompt!</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>hello
</code></pre></div></div>
<p>Did you get your message on the screen? Thrilling, isn’t it??</p>
<p>I hope you enjoyed this retro “Hello world” experience!</p>
<h2 id="bonus---more-on-the-pc-dos--ms-dos-history">Bonus - more on the PC-DOS / MS-DOS history</h2>
<p>If you would like to learn more about the history behind PC-DOS and MS-DOS, I recommend the following videos:</p>
<ul>
<li><a href="https://youtu.be/3E5Hog5OnIM">Why DOS Was (and Is) a Thing</a> - YouTube - 32 minutes long</li>
<li><a href="https://youtu.be/dHR3xoRT-4M">What is IBM PC DOS 2000? - History and Unboxing</a> - YouTube - 26 minutes long</li>
</ul>
<h2 id="references">References</h2>
<ul>
<li><a href="https://en.wikipedia.org/wiki/IBM_Personal_Computer">https://en.wikipedia.org/wiki/IBM_Personal_Computer</a></li>
<li><a href="https://en.wikipedia.org/wiki/Intel_8088">https://en.wikipedia.org/wiki/Intel_8088</a></li>
<li><a href="https://en.wikipedia.org/wiki/IBM_PC_DOS">https://en.wikipedia.org/wiki/IBM_PC_DOS</a></li>
<li><a href="https://en.wikipedia.org/wiki/DOS_API">https://en.wikipedia.org/wiki/DOS_API</a></li>
<li><a href="https://thestarman.pcministry.com/asm/debug/debug.htm">https://thestarman.pcministry.com/asm/debug/debug.htm</a></li>
<li>Emulators, disk images and roms
<ul>
<li><a href="https://www.pcjs.org/machines/pcx86/ibm/5150/mda/">https://www.pcjs.org/machines/pcx86/ibm/5150/mda/</a></li>
<li><a href="https://www.pcem-emulator.co.uk/downloads.html">https://www.pcem-emulator.co.uk/downloads.html</a></li>
<li><a href="https://github.com/BaRRaKudaRain/PCem-ROMs/tree/master/ibmpc">https://github.com/BaRRaKudaRain/PCem-ROMs/tree/master/ibmpc</a></li>
</ul>
</li>
</ul>I like spending some of my time learning more about the architecture of retro computers. That’s an interest that comes from a very early love for computers in my life and that includes video games. From time to time I read about this topics and some time ago I was interested in the IBM PC (IBM 5150), launched in 1981.My first ROM Hack, fixing the infinite magic on Master System’s Ninja Gaiden2021-04-24T00:00:00+00:002021-04-24T00:00:00+00:00https://dellagustin.github.io/videogames/blogging/emulation/rom%20hacking/2021/04/24/my-first-rom-hack<h2 id="tldr">TL;DR</h2>
<p>I have made my first ROM hack, fixing an issue on Master System’s Ninja Gaiden, where get infinite magic points once you collect 999 or more of them.</p>
<p>Here are the BPS patches:</p>
<ul>
<li><a href="/assets/downloads/Ninja-Gaiden-Europe-avoid-infinite-magic.bps">Ninja-Gaiden-Europe-avoid-infinite-magic.bps</a></li>
<li><a href="/assets/downloads/Ninja-Gaiden-Europe-prototype-avoid-infinite-magic.bps">Ninja-Gaiden-Europe-prototype-avoid-infinite-magic.bps</a></li>
</ul>
<h2 id="a-bit-of-my-motivation">A bit of my motivation</h2>
<p>Growing up in Brazil, I owned a Sega Master System when I was kid. The SMS was far more available there, and probably more popular then the Nintendo Entertainment System (NES), as it was very well distributed by a local company, Tec Toy.</p>
<p>I’m a long time gamer, and a software developer by trade, with some knowledge on electronics and retro computers architecture, more theoretical than practical.</p>
<p>Putting all that together, it is not a surprise that I have a certain fascination for retro games and their inner workings.</p>
<p>For quite some time now I have been wanting to get something done in that area, and ROM hacking is a nice way to get acquainted with the topic.</p>
<p>In my spare time I am also an occasional podcaster. At <a href="https://fliperamadeboteco.com/">Fliperama de Boteco</a>, a Brazilian podcast, we talk mostly about retro games. Some time ago we published an <a href="https://fliperamadeboteco.com/2019/05/30/fliperama-de-boteco-179-ninja-gaiden-master-system/">episode on Ninja Gaiden for the Master System</a>. One of things that got our attention was the fact that once you collected more than 999 magic points, you could use special weapons and magic without running out of points, specially with a fire magic that makes you invincible, and normally cost you 50 magic points. That made the game too easy, and to me it felt like a bug, as you could collect enough magic to do this early in the game.</p>
<p>Recently, someone posted about this game in a Brazilian Master System group on Facebook, and one of the replies was “I pray that someone will make a hack to fix the bug of the thousand magic points … it makes the game too easy when it reaches 999”. I took that as an interesting challenge to get myself initiated in ROM hacking, and went for it.</p>
<h2 id="how-i-did-it">How I did it</h2>
<p>The fist step was to find out where in the RAM (Random Access Memory) the information about the number of magic points was stored.</p>
<p>I started searching for it using the emulator BizHawk, which contains a <em>RAM Search</em> function.</p>
<p>Using this feature, I eventually found out that magic points were stored in the following addresses:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">0x1FB8</code> - first digit</li>
<li><code class="language-plaintext highlighter-rouge">0x1FB9</code> - second digit</li>
<li><code class="language-plaintext highlighter-rouge">0x1FBA</code> - third digit</li>
<li><code class="language-plaintext highlighter-rouge">0x1FBB</code> - this one is normally 0, but it is set to 255 (<code class="language-plaintext highlighter-rouge">0xFF</code> in hexadecimal) when you collect more than 999 points</li>
</ul>
<p>For example, if you have 150 magic points, these will be the values stored addresses above:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">0x1FB8</code> - 0</li>
<li><code class="language-plaintext highlighter-rouge">0x1FB9</code> - 5</li>
<li><code class="language-plaintext highlighter-rouge">0x1FBA</code> - 1</li>
<li><code class="language-plaintext highlighter-rouge">0x1FBB</code> - 0</li>
</ul>
<p><img src="/assets/ninja-gaiden-ram-watch-290.png" alt="RAM Watch 290 magic points" /></p>
<p><img src="/assets/ninja-gaiden-ram-watch-999.png" alt="RAM Watch 999 magic points" /></p>
<p>One important information, this addresses are relative to <code class="language-plaintext highlighter-rouge">0x0000</code>, but RAM on the Master System is mapped to the address range from <code class="language-plaintext highlighter-rouge">0xC000-0xDFFF</code> (and mirrored on the address range <code class="language-plaintext highlighter-rouge">0xE000-0xFFFF</code>), so <code class="language-plaintext highlighter-rouge">0x1FB8</code> is actually mapped at <code class="language-plaintext highlighter-rouge">0xDFBB</code> or <code class="language-plaintext highlighter-rouge">0xFFBB</code>.</p>
<p>Now that I have identified these addresses, I started looking for the code that changed their content.</p>
<p>For that I used a different emulator, <a href="https://emulicious.net/">Emulicious</a>, which has a very good disassembler and debugger.</p>
<p>I added breakpoints to stop the execution where this addresses were read or written, and started doing stuff like using magic and getting items that granted magic points, also reaching over 999, then checking the code around the breakpoints.</p>
<p>Ultimately I discovered that <code class="language-plaintext highlighter-rouge">0xDFBB</code> was used to protect the magic counter from overflowing, but I also found out that it was checked at the beginning of routine that subtracted magic points when magic was used:</p>
<p><img src="/assets/ninja-gaiden-emulicious-debugging.png" alt="Emulicious debugging, routine at 0x50D9" /></p>
<pre><code class="language-asm">_LABEL_50D9_:
ld a, (_RAM_DFC3_)
or a
jr nz, _LABEL_513E_
ld a, (_RAM_DFBB_) ; load content of 0xDFBB into register a
or a ; bitwise OR of a with a, store the result in a
jr nz, _LABEL_513E_ ; jump to 0x513E if the result of last operation (a) is not zero
... ; continue to subtraction routine
_LABEL_513E_:
scf
ret
</code></pre>
<p>So I figured that setting the content of <code class="language-plaintext highlighter-rouge">0xDFBB</code> back to zero, instead of checking it and jumping, should give me the result I wanted.</p>
<p>To do that, I used the Hex Editor <a href="https://sourceforge.net/projects/wxhexeditor/">wxHexEditor</a>.</p>
<p>First thing was look at the address <code class="language-plaintext highlighter-rouge">0x50D9</code> (20697 in decimal) and locate the opcodes for the check:</p>
<p><img src="/assets/ninja-gaiden-hexeditor-50D9.png" alt="HexEditor at 0x50D9" /></p>
<p>Now, without calling another routine, I had 6 bytes to do what I wanted, set the content of <code class="language-plaintext highlighter-rouge">0xDFBB</code> with <code class="language-plaintext highlighter-rouge">0</code>.</p>
<p>There are more sophisticated ways of doing this, like using a Z80 assembler like WLA-DX, but that would be like bringing a gun to a knife fight. I did not even had to look for the opcodes of the instructions I wanted to use to replace the check:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> ld a, $00
ld ($DFBB), a
</code></pre></div></div>
<p>there was a very similar piece of code just above the one I wanted to change:</p>
<p><img src="/assets/ninja-gaiden-hexeditor-50D0.png" alt="HexEditor at 0x50D0" /></p>
<p>It was only 5 bytes, I just had to adapt so that it was <code class="language-plaintext highlighter-rouge">ld a, $00</code> instead of <code class="language-plaintext highlighter-rouge">ld a, $FF</code>, and fill the remaining byte with <code class="language-plaintext highlighter-rouge">0</code>, which is the instruction <code class="language-plaintext highlighter-rouge">nop</code> (<em>No Operation</em>):</p>
<p><img src="/assets/ninja-gaiden-hexeditor-50D9-hack.png" alt="HexEditor at 0x50D0, hack" /></p>
<p>Now, I tried to load the game, and…</p>
<p><img src="/assets/ninja-gaiden-patch-with-wrong-checksum.png" alt="Patch with wrong checksum" /></p>
<p>Ooops, something is wrong…</p>
<p>The ROMs have a checksum stored at the address <code class="language-plaintext highlighter-rouge">0x7FFA</code>, which is checked by the BIOS. After hacking the ROM we need to adjust the checksum. To calculate the checksum for the patch I used the command line tool <a href="https://www.smspower.org/Development/SMSCheck">SMS Check</a>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>C:\Emuladores\Roms-SMS>smscheck.exe "Ninja Gaiden (Europe) - avoid infinite magic.sms"
SMS Check 0.24 - by Omar Cornut (Bock), Apr 19 2016
http://www.smspower.org
--
[./Ninja Gaiden (Europe) - patched with failure.sms]
Size ...... 262144
Header .... 54 4D 52 20 53 45 47 41 00 00 AD 4C 01 71 00 40
System .... SMS Export (4)
Size ...... 256k+
CheckSum .. 4BB3, Bad. Should be 4CAD
FullSum ... 1674F91
</code></pre></div></div>
<p>All, right, now I adjusted the checksum in the ROM, again using the hex editor:</p>
<p><img src="/assets/ninja-gaiden-hexeditor-7FFA-hack.png" alt="HexEditor at 0x7FFA, hack" /></p>
<p>And, there we go:</p>
<p><img src="/assets/ninja-gaiden-no-infinite-magic.gif" alt="Ninja Gaiden - no infinite magic" /></p>
<p><strong>No more infinite fire magic!</strong></p>
<p>Now I had my hack ready, so it is time to produce a BPS patch, I used a tool called <a href="https://www.romhacking.net/utilities/1040/">Floating IPS</a>.</p>
<p><img src="/assets/floating-ips.png" alt="Floating IPS" /></p>
<p>And there we have it, a patch file ready to be applied: <a href="/assets/downloads/Ninja-Gaiden-Europe-avoid-infinite-magic.bps">Ninja-Gaiden-Europe-avoid-infinite-magic.bps</a></p>
<p>Later I was informed that there was also a prototype ROM for Ninja Gaiden on the Master System, so I produced a patch for that one too: <a href="/assets/downloads/Ninja-Gaiden-Europe-prototype-avoid-infinite-magic.bps">Ninja-Gaiden-Europe-prototype-avoid-infinite-magic.bps</a></p>
<h2 id="closing-words">Closing words</h2>
<p>It was a fun ride, getting acquainted with Z80 assembly takes a bit of patience, my last contact with assembly was around 20 years ago… managing to read it after sometime is quite satisfactory. I hope this blog post helps other people that would like to get started with ROM hacking.</p>
<p>A huge thanks go to the people maintaining the excellent content on the website smspower.org.</p>
<h2 id="references">References</h2>
<ul>
<li><a href="https://en.wikipedia.org/wiki/Ninja_Gaiden_(Master_System)">https://en.wikipedia.org/wiki/Ninja_Gaiden_(Master_System)</a></li>
<li><a href="http://www.romhacking.net/forum/index.php?topic=19594.msg276409#msg276409">http://www.romhacking.net/forum/index.php?topic=19594.msg276409#msg276409</a></li>
<li><a href="https://landley.net/history/mirror/cpm/z80.html">https://landley.net/history/mirror/cpm/z80.html</a></li>
<li><a href="https://wikiti.brandonw.net/index.php?title=Z80_Instruction_Set">https://wikiti.brandonw.net/index.php?title=Z80_Instruction_Set</a></li>
<li><a href="https://www.smspower.org/Articles/TrainersDoItYourself">Trainers - Do It Yourself</a> at smspower.org</li>
<li><a href="https://www.smspower.org/Development/InstructionSet">Z80 Instructions Set</a> at smspower.org</li>
<li><a href="https://www.smspower.org/Development/ROMHeader#Checksum7ffa2Bytes">ROM Header, Checksum</a> at smspower.org</li>
<li><a href="https://www.smspower.org/Hacks/HowToUseBPSAndIPSPatchFiles">How to Use BPS and IPS Patch Files</a> at smspower.org</li>
<li><a href="https://www.romhacking.net/utilities/1040/">Floating IPS</a> at romhacking.net</li>
</ul>TL;DRHow to create GIFs and MP4 videos from NES emulator captures2018-09-04T20:40:00+00:002018-09-04T20:40:00+00:00https://dellagustin.github.io/videogames/blogging/emulation/2018/09/04/how-to-create-gifs-and-mp4s-from-nes-emu-captures<h2 id="intro-and-some-background-jump-over-for-the-how-to">Intro and some background (Jump over for the how to)</h2>
<p>I recently started collaborating with a Brazilian website and podcast called
<a href="http://fliperamadeboteco.com" target="_blank">Fliperama de Boteco</a>. The name refers to
a video arcade that happened to be in a low level type of bar (common on our late
80s and through the 90s) and the website is mainly focused on retro gaming.</p>
<p>I wanted to write an analysis on the game <em>Vice: Project Doom</em> for the NES and
put some GIFs to illustrate parts of the gameplay, <a href="http://fliperamadeboteco.com/vice-project-doom-descubra-um-classico-obscuro-do-nes/">here is the result</a>
(in portuguese, but you can see the GIFs and videos).</p>
<p>Beforehand it sounded like an easy peasy tasks. It is 2018, GIFs are everywhere
(as if it was the early 2000s), no big deal.</p>
<p>Well, It is easy, if you know where to look for, and you know some tips and tricks.</p>
<p>I did not record all my failed attempts, but I tested many software before reaching
what I consider a working confortable solution.</p>
<p>See below how I did.</p>
<h2 id="how-to">How to</h2>
<p>Here is what you will <strong>need</strong>:</p>
<ul>
<li>The NES emulator <a href="http://www.fceux.com">FCEUX</a></li>
<li>The <a href="https://lags.leetcode.net/codec.html">Lagarith</a> lossless video codec</li>
<li>The <a href="https://www.ffmpeg.org/">FFmpeg</a> software (In our tutorial, we’ll run it on <a href="https://www.docker.com/">Docker</a>)</li>
<li>Optionally, since we are going to use FFmpeg on the command line, I recommend to use <a href="https://github.com/cbucher/console">ConsoleZ</a></li>
</ul>
<h3 id="do-your-nes-captures">Do your NES captures</h3>
<p>Fire up the FCEUX emulator, load your ROM, play until the part you want to record.
Now simply go to the menu an select <code class="language-plaintext highlighter-rouge">File > AVI/Wav > Record AVI...</code>, select the path
where to save your file and when prompted for a codec select <code class="language-plaintext highlighter-rouge">Lagarith Lossless Codec</code> in the drop down.</p>
<p>Now play until you are finished (you can you the pause key to pause the recording), and select in the menu
<code class="language-plaintext highlighter-rouge">File > AVI/Wav > Stop AVI</code>.
That’s it, your capture is done.</p>
<h3 id="generate-a-gif-from-a-capture">Generate a GIF from a capture</h3>
<p>Now, based on this nice <a href="http://blog.pkh.me/p/21-high-quality-gif-with-ffmpeg.html">blog post</a>, I adapted the script <code class="language-plaintext highlighter-rouge">convert_to_gif.bat</code>
to run FFmpeg from Docker container (because I don’t like installing stuff):</p>
<div class="language-bat highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">SET</span> <span class="kd">palette</span><span class="o">=</span><span class="kd">tmp</span><span class="na">/palette</span>.png
<span class="kd">SET</span> <span class="kd">filters</span><span class="o">=</span><span class="kd">fps</span><span class="o">=</span><span class="m">50</span>
<span class="kd">ECHO</span> <span class="s2">"Generating palette"</span>
<span class="kd">docker</span> <span class="nb">run</span> <span class="na">--rm -v </span><span class="nv">%CD%</span>:/data <span class="kd">jrottenberg</span><span class="na">/ffmpeg -v </span><span class="kd">warning</span> <span class="na">-i /data</span>/<span class="err">%</span><span class="m">1</span> <span class="na">-vf </span><span class="s2">"</span><span class="nv">%filters%</span><span class="s2">,palettegen"</span> <span class="na">-y /data</span>/<span class="nv">%palette%</span>
<span class="kd">ECHO</span> <span class="s2">"Generating gif"</span>
<span class="kd">docker</span> <span class="nb">run</span> <span class="na">--rm -v </span><span class="nv">%CD%</span>:/data <span class="kd">jrottenberg</span><span class="na">/ffmpeg -v </span><span class="kd">warning</span> <span class="na">-i /data</span>/<span class="err">%</span><span class="m">1</span> <span class="na">-i /data</span>/<span class="nv">%palette%</span> <span class="na">-lavfi </span><span class="s2">"</span><span class="nv">%filters%</span><span class="s2"> [x]; [x][1:v] paletteuse=diff_mode=rectangle"</span> <span class="na">-y /data</span>/<span class="err">%</span><span class="m">2</span>
</code></pre></div></div>
<p>You can use it like: <code class="language-plaintext highlighter-rouge">convert_to_gif.bat nes-capture.avi nes-capture.gif</code></p>
<p>One important remark, I set the output framerate to 50 FPS, even though I captured at 60fps.<br />
The reason is that browsers are not very good at rendering GIFs with very small delay between
frames. After some tests I reached the conclusion 50 FPS was a good number as it was well
rendered in Edge and Chrome.</p>
<p>See the result below:</p>
<table>
<thead>
<tr>
<th>50 FPS - size: 47KB</th>
<th>60 FPS - size: 53KB</th>
</tr>
</thead>
<tbody>
<tr>
<td><img src="/assets/Vice - Project Doom (USA)-Moves-50fps.gif" alt="GIF at 50 FPS" /></td>
<td><img src="/assets/Vice - Project Doom (USA)-Moves-60fps.gif" alt="GIF at 50 FPS" /></td>
</tr>
</tbody>
</table>
<h3 id="generate-a-mp4-video-from-a-capture">Generate a MP4 video from a capture</h3>
<p>Again using Docker, I wrote this one line script <code class="language-plaintext highlighter-rouge">convert_to_mp4.bat</code>:</p>
<div class="language-bat highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">docker</span> <span class="nb">run</span> <span class="na">--rm -v </span><span class="nv">%CD%</span>:/data <span class="kd">jrottenberg</span><span class="na">/ffmpeg -i /data</span>/<span class="err">%</span><span class="m">1</span> <span class="na">-map </span><span class="m">0</span>:0 <span class="na">-vf </span><span class="nb">format</span><span class="o">=</span><span class="kd">yuv420p</span> <span class="na">-profile</span><span class="nl">:v</span> <span class="kd">baseline</span> <span class="na">-level </span><span class="m">3</span>.0 <span class="na">-preset </span><span class="kd">veryslow</span> <span class="na">-r </span><span class="m">30</span> <span class="na">/data</span>/<span class="err">%</span><span class="m">2</span><span class="sb">`
</span></code></pre></div></div>
<p>Explaining the parameters a bit:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">-map 0:0</code> - Filters only track 0, which means only the video, no audio. Remove if you want audio</li>
<li><code class="language-plaintext highlighter-rouge">-vf format=yuv420p</code> - I don’t even know how to explain, but it does not work without it</li>
<li><code class="language-plaintext highlighter-rouge">-profile:v baseline -level 3.0</code> - See https://trac.ffmpeg.org/wiki/Encode/H.264#Compatibility</li>
<li><code class="language-plaintext highlighter-rouge">-preset veryslow</code> - Best compression, takes a bit more time, see https://trac.ffmpeg.org/wiki/Encode/H.264#Preset</li>
<li><code class="language-plaintext highlighter-rouge">-r 30</code> - Output at 30 FPS. It is sad, but certain devices still do not support HTML5 video at 60 FPS :(</li>
</ul>
<p>You can use it like: <code class="language-plaintext highlighter-rouge">convert_to_mp4.bat nes-capture.avi nes-capture.mp4</code></p>
<p>See the result below:</p>
<table>
<thead>
<tr>
<th><code class="language-plaintext highlighter-rouge">-profile:v baseline -level 3.0 -r 30</code> <br /> size: 187KB</th>
<th><code class="language-plaintext highlighter-rouge">-profile:v high -level 4.2</code> <br /> unchanged framerate (~60FPS) <br /> size: 133KB</th>
</tr>
</thead>
<tbody>
<tr>
<td><video src="/assets/Vice - Project Doom (USA)-baseline-3.0-30fps.mp4" controls="" autoplay="" loop=""></video></td>
<td><video src="/assets/Vice - Project Doom (USA)-high-4.2-60fps.mp4" controls="" autoplay="" loop=""></video></td>
</tr>
</tbody>
</table>
<h2 id="final-considerations">Final Considerations</h2>
<p>At the end I used both GIFs and videos in my blog post.<br />
Both have their pros and cons.</p>
<p>GIFs can achieve a higher framerate (up to 50 FPS) in most devices and can produce smaller
files depending on the type of action you have in the video.</p>
<p>Videos will produce smaller files if you have a lot of action going on, but you will have to lower
the framerate if you want more compatibility with older devices.</p>
<p>My tip is to always test both and see which one fits best to your needs.</p>Intro and some background (Jump over for the how to)