Bootloaders

lowpoly · 12 Mar 2014, 17:24

(edited with current information from thread)

Hi,

I'm currently "evaluating" (i.e. googling) bootloader options for the Atmega32u4 and maybe the At90usb1286.

I found 9 so far:

AVRUSBBoot, 2KB, proprietary open source license (by Objective Development), C++ source for loader, requires driver on Windows?, V-USB based (low speed 1.5MHz)
LUFA HID, 4KB, Open Source, works with the modified Teensy CLI which is also open source.
LUFA DFU, 4KB, works with Atmel FLIP or dfu-programmer, which is open source
LUFA CDC, size?, works with AVRDUDE
USBaspLoader, 2KB, works with AVRDUDE, GNU GPL, V-USB based (low speed 1.5MHz)
BootloadHID, 2KB, comes with command line tool, separate GUI available, V-USB based (low speed 1.5MHz)
Atmel DFU loader, 4k, requires FLIP which is not available for OSX (or dfu-programmer?), closed source
Arduino Leonardo bootloader, 4k, open source, requires Arduino envrionment?
Teensy Halfkay, 0.5kB, only available with Teensy, Teensy Loader or CLI, closed source (not the CLI)

Not sure if the Objective Development loaders (USBaspLoader and BootloadHID) require V-USB.

So far the LUFA HID seems to be the most attractive. With BootloadHID on second (in case V-USB is not required) and AVRUSBBoot being third.

Am I overlooking something? Experiences?

I'm also not sure if the Atmel DFU loader comes preinstalled on new ATmegas. But the programming options seem to lack simplicity.

matt3o · 12 Mar 2014, 17:54

This is very interesting to me. What would be the advantage of one bootloader instead of another? Also, how do you upload a different bootloader?

PS: arduino bootloader works with AVRDUDE

scottc · 12 Mar 2014, 17:59

I've flashed an Arduino Leo and a Pro Micro clone both using AVRdude too. The Pro Micro clones are great because they're about €2.50. Their drawback is that it's difficult to get into their bootloaders...

lowpoly · 12 Mar 2014, 18:23

The Arduino bootloader was the only one where I heard it should be avoided. I think it gave problems on the second write.

I plan to use AvrStudio and the AvrISP mkII Programmer. Seemed to be the simplest solution.

4K vs. 2K would be an issue.

There should also be some simple approach (for ex. a single source code file, like Teensy CLI) on the host side.

kile · 13 Mar 2014, 08:39

USBaspLoader, AVRUSBBoot and BootloaderHID are all V-USB based (low speed 1.5MHz). This means they are meant to be used on AVRs that don't have hardware USB (all AVRs with names not ending with u2, u4 or begining with at90usb). ATmega32u4 has hardware USB (full speed 12MHz) so using V-USB on these is possible, but kinda pointless.

Teensy Halfkay is not only closed source but also unavailable - you can't download the Halfkay bootloader HEX file anywhere. It's PJRC's "trade secret" and only PJRC can burn a Halfkay bootloader into an AVR.

I'd go with LUFA HID. No drivers are needed for windows, and there is an open source command line program for the PC side. I've made a Windows GUI for it a while ago, but never used it. It should be working, let me know if you need it and I'll clean it up and upload it somewhere.

hasu · 13 Mar 2014, 08:51

USBaspLoader is very useful on Mega AVR, I used it for my keyboard project and it worked perfectly. You can use V-USB bootloaders on USB AVR like ATMega32U4 and AT90USB but it is virtually useless that you use V-USB instead of hardware USB support the chips have.

After I play with a Teensy for a while I've used stock Atmel DFU loader with dfu-programmer on Windows and Linux for a year or two. No problem except for its size. It is a factory default bootloader you don't need to flash to get the bootloader. This is a good score.

Yes, with Leonardo bootloader I also had very bad experience, even with Arduino IDE. I would replace it with LUFA DFU bootloader if I get those cheap compatible boards. LUFA DFU bootloader worked flawlessly for me with dfu-programmer and it looks compatible with Atmel DFU.

Halfkay and HID based bootloaders don't required driver install this will be good point as well as its size. But I don't have a experience with these bootloaders except for Teensy.

hasu · 13 Mar 2014, 08:56

How difficult is writing USB bootloader within 512 bytes? It looks to me PJRC HalfKay is a magic.
A decent engineer can do this job? We need a Wizard?

kile · 13 Mar 2014, 09:12

hasu wrote:How difficult is writing USB bootloader within 512 bytes? It looks to me PJRC HalfKay is a magic.
A decent engineer can do this job? We need a Wizard?

Pretty much. Even a 1K USB bootloader would be a stretch. Halfkay is written in AVR assembler and you would need to be a seasoned assembler programmer to pull it off. I've never done any assembler, but I know enough to imagine how hard it would be.

lowpoly · 13 Mar 2014, 12:24

Thanks for the replies.

I agree that V-USB wouldn't make much sense on a hardware USB chip.

kile wrote:I'd go with LUFA HID. No drivers are needed for windows, and there is an open source command line program for the PC side. I've made a Windows GUI for it a while ago, but never used it. It should be working, let me know if you need it and I'll clean it up and upload it somewhere.

That's what I'll try then. I'll get back to you about the Windows GUI when the Miniguru is finished, thank you. I still have to do this some day (if it will fit, can be unbranded too).

jdcarpe · 13 Mar 2014, 14:21

FYI, PJRC sells ATmega32U4 in TQFP-44 package with the HalfKay bootloader programmed onto it for $8.

http://www.pjrc.com/store/ic_atmega32u4_tqfp.html

The thing I like most about the HalfKay bootloader is that I've never failed to flash a firmware using the Teensy Loader software. Using FLIP is a real annoyance.

I have also had much success on Linux using the dfu-programmer with the default Atmel DFU bootloader.

I was able to flash a Pro Micro using avrdude, only after MULTIPLE attempts to write the FW.

scottc · 13 Mar 2014, 14:53

jdcarpe, I think I found the root of our Pro Micro woes - in order to get into the bootloader, you need to short the RST pin _twice_ as it starts up. Soarer and someone else (can't remember who, sorry!) on IRC helped me to figure this out.

jdcarpe · 13 Mar 2014, 15:25

Aha! Well, that is good information for my next attempt. Thanks, scottc (and Soarer et al)!

lowpoly · 22 Mar 2014, 20:55

The LUFA HID bootloader also needs 4K. Here it compiles to 2222 bytes code and data so it is very close to 2K but close is not enough.

kile · 23 Mar 2014, 01:24

Which version of avr-gcc did you compile it with?
Try adding a -flto to compilation if your avr-gcc version is at least 4.7

Grendel · 23 Mar 2014, 08:16

kile wrote:
hasu wrote:How difficult is writing USB bootloader within 512 bytes? It looks to me PJRC HalfKay is a magic.
A decent engineer can do this job? We need a Wizard?
Pretty much. Even a 1K USB bootloader would be a stretch. Halfkay is written in AVR assembler and you would need to be a seasoned assembler programmer to pull it off. I've never done any assembler, but I know enough to imagine how hard it would be.

I got annoyed w/ Flip when I was debugging a 40% board running on a 32U4 so I reverse engineered the Teensy BL protocol and wrote a loader from scratch, emulating it in order to use the Teensy app. Since I needed it fast I stuck to C, it came out bigger than 512b but less than a k. I probably could pull the 512b of by switching to 100% assembler, but I won't do it -- takes time and I'm fine w/ a 1k loader plus I don't want to get on the wrong side w/ Paul (of PJRC). I still need Teensy's in the future...

Although I have been using it for that 40% board, it's not tested beyond that. I've attached the hex image in case someone would like to take it for a spin (and please report back if you have problems.) Please use at your own risk!

(Since the hex files aren't allowed as attachment (why?), here's the contens. Just copy and save it as OneKay.hex)

Edit: forgot -- this is for an ATMega32U4 w/ a 16MHz oscillator, fuses I'm using it w/ are: 0:0x5E 1:0xDD 2:0xF1

Spoiler:

kile · 23 Mar 2014, 09:09

Well, congratulations to you, Grendel! You made that MS Sidewinder USB converter too, right?

Grendel · 23 Mar 2014, 09:22

Why, indeed I did.

On a side-note -- OneKay probably can be made to work w/ a variety of AVR chips w/ the U2 and U4 USB modules. For AVR's w/ more than 64k flash it needs some additional work...

kile · 23 Mar 2014, 13:48

lowpoly wrote:The LUFA HID bootloader also needs 4K. Here it compiles to 2222 bytes code and data so it is very close to 2K but close is not enough.

I checked and it turns out -flto has no effect at all. But if you do what Dean (author of LUFA's) suggests here https://groups.google.com/forum/#!topic ... q4RTLqiMRA you will get bellow 2k. I have just tried it with avr-gcc 4.8.1 (which you can get in the latest AVR toolchain from Atmel) and it went from 2092 down to 1966 bytes.

grendel wrote:I probably could pull the 512b of by switching to 100% assembler, but I won't do it -- takes time and I'm fine w/ a 1k loader plus I don't want to get on the wrong side w/ Paul (of PJRC). I still need Teensy's in the future...

Now that I think about it, I think you are more likely to get on Paul's wrong side if you write a bootloader that can use the TeensyLoader than if you get below 512b. I think the most appealing feature for the majority of the Teensy users is the loader PC program and not the fact that the bootloader fits in 512b. The loader's GUI has large, colourful bitmapped buttons and there aren't any confusing options to select. And for how many will the extra 1.5k or 0.5k really make a difference?

Besides, getting on Paul's wrong side already happened with LUFA and the TeensyHID. And that was with a 2K bootloader, imagine how happy he will be with 1k?

But to be fair, Paul's hard work has helped a lot of people get projects up and running and he should be rewarded for it. The entire mechanical keyboard community probably wouldn't be half as big if it wasn't for the Teensy.

Grendel wrote:Why, indeed I did.

That was a nice project, btw. I found it about the time I started playing with AVRs. Learned quite a few things from it too. Thanks for that!

lowpoly · 23 Mar 2014, 18:17

kile wrote:Which version of avr-gcc did you compile it with?

gcc 4.7.2 and libc 1.8.0.

Edit: Updated to 4.8.1 but got the same size (2222). Now to the optimizations.

kile wrote:I checked and it turns out -flto has no effect at all. But if you do what Dean (author of LUFA's) suggests here https://groups.google.com/forum/#!topic ... q4RTLqiMRA you will get bellow 2k. I have just tried it with avr-gcc 4.8.1 (which you can get in the latest AVR toolchain from Atmel) and it went from 2092 down to 1966 bytes.

Thank you. Will try that now.

kile wrote: I think the most appealing feature for the majority of the Teensy users is the loader PC program and not the fact that the bootloader fits in 512b.

Paul's command line interface is under the GNU GPL and the modified version still contained in LUFA. Converting that to a windowed application should be trivial (and legal)?

@Grendel: 1K, wow.

kile · 23 Mar 2014, 19:31

lowpoly wrote:Paul's command line interface is under the GNU GPL and the modified version still contained in LUFA. Converting that to a windowed application should be trivial (and legal)?

Sure it is, that's what I did for that Win GUI I mentioned. But doing the same for Win/Linux/Mac? And making it look and work identical on all three? And it has to be tested and it has to work on all the available USB host controllers and all OS versions. That is not as trivial, and certainly not as much fun. Sure, it is possible and doable. But otherwise... naaah..

lowpoly · 23 Mar 2014, 20:03

I removed USB_Device_ClearSetFeature() (function and call) from DeviceStandardReq.c and get this size:

Program: 2004 bytes (6.1% Full)
(.text + .data + .bootloader)
Data: 86 bytes (3.4% Full)
(.data + .bss + .noinit)

Looks like I'm good. I had wrongly assumed that Data would also be stored in the 2k bootloader space. So, before it was 2136 bytes, not 2222.

The AvrISP is in the shop, testing tomorrow.

lowpoly · 23 Mar 2014, 20:13

kile wrote:Sure it is, that's what I did for that Win GUI I mentioned. But doing the same for Win/Linux/Mac? And making it look and work identical on all three? And it has to be tested and it has to work on all the available USB host controllers and all OS versions. That is not as trivial, and certainly not as much fun. Sure, it is possible and doable. But otherwise... naaah..

Ah, thanks. Looks like I completely underestimated this so far.

lowpoly · 24 Mar 2014, 10:51

lowpoly wrote:The AvrISP is in the shop, testing tomorrow.

Tested and it seems to work. No complaints during flashing this time, fuses set to 2K (1K words), makefiles of bootloader and firmware set to 2K. I think the last one is only for displaying correct sizes after linking.

Grendel · 24 Mar 2014, 19:01

On a sidenote, I remember at some point in time I used BootloadHID for an earlier version of the 3DPro converter. Should be easy to port to the U2/U4 modules, I'll take a look. May be able to squeeze that into 1k as well

Edit: would probably end up w/ 2k, but doable. I'll take a closer look tonight.

Grendel · 25 Mar 2014, 04:58

I got the framework done but the core will take longer than an evening. I'll put that on the back-burner unless there's some interest in having BootloadHID for AVR's w/ U2/4 USB modules.

Grendel · 26 Mar 2014, 07:13

Well, guess it took two evenings

FWIIW, I got the BootloadHID compatible loader working, tested w/ the above mentioned 32U4. Adding the necessary string descriptors and report1 data pushed the size above 1k again, so it chews 2k of the flash (still better than Atmel's 4k loader..) Maybe one day I'll replace the libc vector table w/ a custom one, should bring it back to 1k..

Again, if someone wants to take it for a spin... Please report back if you do. Image is for a ATMega32U4, 16MHz oscillator, fuses as above but w/ 2kb boot section. Ask if you want the source.

Spoiler:

kile · 26 Mar 2014, 07:49

Grendel wrote:Image is for a ATMega32U4, 16MHz oscillator, fuses as above but w/ 2kb boot section. Ask if you want the source.

Me, me!

Thanks!

Grendel · 26 Mar 2014, 09:13

Give me another evening, just finished hooking in a custom vector table, but I need to test that and I'm about ready to go to bed

If this works (and is reasonably safe across different CPU's) it'll bring the size down to < 1k again.

Grendel · 27 Mar 2014, 04:08

Here we go. BootHID v02 for 1k boot sections can be found at drive.google.com/ under file{slash}d/0ByFalc_WhYoscVphMmxhRnFLVXM/edit{quest.mk.}usp=sharing (sorry, you will have to assemble that link yourself, I don't want bots to easily pick up on it. Mind the text in {}.)

A few words (of mostly warnings to scare you

) -- I tested the code with a Mega32U4 @ 16MHz w/ the HWBE fuse programmed. The AVRS project is configured for this and the hex file is included. I used AVR Studio 4.19 730 (since I don't need the added features of later versions yet and it's less of a hog..), so the included project file is from that version. ISP used is an Atmel AVRISP mkII. Before doing anything you should read the (lengthy, sorry) Release Notes and the beginning of usb_hid.c, it explains how to reconfigure the project for different chips. Note that I used avr-libc 1.8.0, the linker scripts may not work w/ older (or possibly newer) versions!

Chips supported (in theory..) are: ATMega[8U2 | 16U2 | 32U2 | 16U4 | 32U4] and AT90usb[82 | 162 | 646 | 1286] at 8 or 16MHz -- that makes 18 possible configurations times two possible invocation methods. I tested one so far..

Naturally I would be interested in feedback if someone uses an untested configuration.

What else.. the black magic level of the code is medium, don't change things around arbitrary, it will break! Esp. don't add any code in main() after the call to usb_init() and don't enable any other interrupts w/o exactly knowing how the code works and what else to change -- you will bring unending misery upon you!

Ah, yes, of course -- use of this code is at the sole responsibility of yourself, I can not hold responsible for anything that happens while it is active !

Addendum: If you want to invoke the boot loader from w/in your application (because the user pressed the forbidden key combination eg.) I would suggest to use the jmp_bootloader() macro found in includes.h -- it'll look at the BOOTSZ fuses and thus work with any size boot loader.

Edit: updated to v02. Forgot to leash the watchdog...

Grendel · 27 Mar 2014, 20:30

Updated BootHID link to v02.

Edit: tested a 2nd configuration: ATMega32U2 @ 16MHz (Frosty Flake), works ! Found a typo in the Release Notes:

Code: Select all

 *	   "-T ..\avr5.x"  - ATMega[16U4|32U4], AT90usb646
 *	   "-T ..\avr51.x" - AT90usb1286
 *	   "-T ..\avr35.x" - ATMega[8U2|16U2|32U2], AT90usb[82|162]

Should read:

Code: Select all

 *	   "-T ..\avr5b.x"  - ATMega[16U4|32U4], AT90usb646
 *	   "-T ..\avr51b.x" - AT90usb1286
 *	   "-T ..\avr35b.x" - ATMega[8U2|16U2|32U2], AT90usb[82|162]