Microchip PICkit3, Don't Halloo Till You Are Out of the Wood

Microchip has a tool called MDB: Microchip command-line debugger. It's a very nice idea: we can automatize things; at least, we can program a hex from a script. Implementation, however, is far from perfect, but, well, it works somehow.

It allows us to create a text file like this:

device PIC18F87K90
hwtool ICD3 -p
program "/path/to/file.hex"
quit

And run mdb, passing a path to this file. However, it doesn't work flawlessly: the first thing that I ran into is that ICD3 warns you all the time:

CAUTION: Check that the device selected in MPLAB IDE (PIC24FV32KA304) is the same one that is physically attached to the debug tool. Selecting a 5V device when a 3.3V device is connected can result in damage to the device when the debugger checks the device ID. Do you wish to continue?

And I have to type y and hit Enter. In recent versions of MDB we can suppress this with the following command:

set system.disableerrormsg "true"

It does disable this message, but sadly it also disables other, pretty regular log messages, which I don't want to disable.

And secondly, it doesn't allow us to select hardware tool by serial number: we have to issue the hwtool command, look at the output which looks like this:

index    Description
0       MPLAB ICD3 tm (JIT140210129)
1       MPLAB ICD3 tm (BUR133553613)

Grab the index, and use it like this: hwtool ICD3 -p 1. It's not convenient at all.

Long story short, I've written a simple python script that runs mdb as a child process, listens for its stdout, issues commands, including the y as an answer to the warning, and so on. I find it much more flexible than the plain mdb. So, when I need to program some hex, my command looks like:

python ./mdb_flash.py --hex="/path/to/file.hex" --mdb-path="/opt/microchip/mplabx/v3.10/mplab_ide/bin/mdb.sh" --mcu="PIC18F87K90" --hwtool="ICD3" --hwtool-serial="JIT140210129"

I like it. You can get the mdb_flash.py script in this GitHub repo.

And recently, I needed to program hex not with ICD3, but with PICkit3. No problem, I've just replaced ICD3 with PICkit3, and… Device which is programmed with exactly the same hex behaves in some weird way. I re-checked it: programmed with ICD3 again, and it works as expected. Program with PICkit3, and it doesn't work normally. Weird.

Okay, what would you do? I'm sure you'd do the same as I did: read hex after programming by ICD3 and PICkit3, and compare two resulting files. Yes, there are differences. Different part is:

Programmed by ICD3:

:0200000400F00A
:1000000001312E322E3300342E35360000B5050076
:1000100000B5050000240100002401E5FFB505003E
:1000200000000000F40100000000000000000000DB
:1000300000000000000000000000000000F40100CB
:1000400000000000000000000000000000000000B0
:100050000000000000F401FFFFFFFFFFFFFFFFFFB4
:10006000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFA0

Programmed by PICkit3:

:0200000400F00A
:10000000000000000000000000000000000000FFF1
:10001000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0
:10002000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFE0
:10003000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFD0
:10004000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFC0
:10005000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFB0
:10006000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFA0

Just in case the reader isn't familiar with the hex format: the command :02 000004 00F0 0A sets the most significant address word to 00F0, while other commands, like 10 0000 00 01312E322E3300342E35360000B5050076 write given data at 00F0 + specified address (in this case, 0000).

It's very interesting: what are the addresses F0xxxx? The first thing to do is to examine the MCU datasheet, especially the section “Memory Organization”, but I failed to find anything there. Was very disappointed by that fact, by the way. Probably I should have tried harder.

Anyway, I found the answer in different way: it's clear that this address is some kind of “special” address, not the program memory, as regular program memory starts with address 0, and we don't have that much memory. It also doesn't look like the configuration bits, since they have different addresses. So, the only thing left is an EEPROM. So, from MPLABX, I've read the EEPROM memory to separate file, and yes, it matches the data from the “correct” hex. Ok, it's quite something!

Here I should say that my device checks the EEPROM memory at startup: if it is all filled with 0xff, then it gets populated with some initial values. If it is already “dirty”, it is left untouched, of course. (Yes, it's better to have checksums, but for this particular project, we keep things simple, at least for now)

So, we have the following: when programmed with PICkit3, the EEPROM memory contains some zeros, for some reason. When the program starts, it reads EEPROM data, which is “dirty”, so, it is left as it is. Later, this garbage data is interpreted somehow, and device behaves weirdly.

Okay, what's next? Why is PICkit3 so bad, and how can we fix it?

I tried to examine MDB docs, and I found that we can explicitly set EEPROM to be programmed by the command set memories.eeprom true. I believe it should be set by default, but anyway, I tried that: I opened mdb manually (instead of automatic script), typed all needed commands, including the set memories.eeprom true, and… It worked!

Full of hope, I've updated my script by adding set memories.eeprom true there, run it, and, damn, it doesn't work.

But it just worked in my manual mdb session! Well, maybe set memories.eeprom true has nothing to do with it? Let's try it: open manual session again, type stuff without setting memories.eeprom, and it worked again. Hmmm. So, it's a red herring.

Then I tried to use “native” mdb scripting facilities: I've created a text file like in the beginning of this article, just changed ICD3 to PICkit3, tried it, and it doesn't work. So, it works when we type it manually, but doesn't work from script.

The only difference that I can think of is the timings. Sounds very weird, but what else can I do?

Yes, in my python script, I've added an artificial 1-second timeout before each command to mdb. Tried it, and it worked! Yahoo!

The next step is, of course, to check where exactly do we need for timeout. After quick experiments, I found out that we need for timeout right after the program command, before the quit command.

So, it turns out that when the program command reports that Program succeeded, it's not quite succeed yet. If we issue the quit command immediately, we end up with corrupted memory. And if we wait a bit, then it works. Ahh, Microchip. Don't halloo till you are out of the wood!

If we want to use standard mdb “scripts”, i.e. text files with commands, we can just add sleep 500 after the program command. Sad but true.

Again, you can get the script here.