Saturday, August 14, 2010

Last month's hacking report

Today's blog entry will be a long one as it will span my activities over the last month. I focused more on hacking than on blogging, but the time has come and the FIFO got filled so it's time to empty it.

Voipac PXA270
Firstly, I got a bit involved with Android. Radoslav Deak and me did some kernel and userland hacking on it to get it running on the Voipac PXA270 board. The Android kernel changes didn't make me feel well, but the userland made me cry, no more words needed here I believe. Anyway, the Android does not support 18bpp LCD, therefore I was told they'll send me another board with 16bpp LCD. Though a mistake happened and I got a board with 18bpp LCD again. Luckily enough, I don't have to hack on Android anymore, but it's not only Android that doesn't support 18bpp LCD, it's also X.org.

Here you'll probably already start questioning my sanity when I say I started digging in the X.org to get this fixed. The X.org developers are actually very nice and very helpful people and they helped me with patching the X.org. One thing worth noting is that I compiled the X.org on the other Voipac PXA270 board which is running Debian already. The building took a little bit of time anyway. The patch is already in mainline X.org I believe and it was not all that hard in the end.

The only trouble with this change is that it uses four bits of memory for each pixel and with the 104MHz SDRAM on the Voipac PXA270 and 640x480 LCD, this eats a lot of memory bus bandwidth.

One more thing is worth noting, that is the xserver-xfbdev, which is capable of 18bpp already, but it's not certain whether the apps connecting to this kind of Xserver won't have problems with 18bpp. On the other hand, this kind of problem may happen even with the standard Xserver, but so far, GDM worked well (and probably whole GTK does, I didn't test anything but that).

There were some other minor changes on the Voipac PXA270 front, but these were mostly bugfixes. There were also bootloader related changes and fixes though. The most relevant one was the one adding support for 128MB variant of the module. All these changes can be found either in my kernel tree or in my u-boot tree.

The OpenBSD
Here comes one sad piece of news. Sad, because I had to leave a piece of code unmaintained. I won't disclose the reasons why as I believe interested people will be able to find it themselves. To put it short, I'm not a member of the OpenBSD developer community anymore. I released the rest of the patches for Palm port to their mailing list and I hope jasper@ and drahn@ will at least not let the code bitrot.

On the other hand, it's actually quite a relief as I can now fully focus on Linux kernel hacking and U-Boot bootloader hacking. My opinion of the OpenBSD kernel code is biased and therefore you won't find it here. Though I think drahn@ did a very good job on the ARM code.

Well, at least I managed to finish my thesis and the system actually works quite well on Palm hardware, including Xenocara (the OpenBSD version of X.org), touchscreen etc.

Palm hacking again
A lot of redundant code in the Palm hardware support code and Eric's whining on that matter some time ago motivated me to consider trimming it a bit. Though hacking on the Palm hardware was different from the old Hack&Dev days this time.

I introduced a set of common functions which each registered one piece of hardware on the Palm handheld in question. This allowed me to remove about six copies of redundant data structures and other stuff. While at this, I wrote the common functions modular so in case the support for a particular piece of hardware isn't compiled in kernel, the support structures are removed as well.

I also added a long lost patch for the MAX1586A PMIC into the common code as that's used in all Palms while at that. Further on, I modularized the rest of the non-common code in all the Palm devices in the same fashion. I'm actually quite happy about the code quality now. And Eric did his little Linus-like gag about the number of deletions ;-)

This was not all on the Palm front though. As my Palm Tungsten|C is a nice toy I like quite a bit and I needed a PXA255 testing platform, I updated the U-Boot on this device to mainline version. This can be found in my u-boot tree again. I also modularized the kernel code the same way I did with PXA27x Palms, though I don't use the common code as this platform is quite different. While at that, I added support for LEDs and switched the UDC to gpio-vbus. All these changes can be found in my kernel tree.

U-Boot
I already said this earlier probably, but I'm now the U-Boot-PXA maintainer. Therefore support for some new machines hit mainline and support for a few others is underway. This time, the new ones are surprisingly the Palm LifeDrive, Palm Tungsten|C and Balloon3. The Palm Tungsten|C is no news as the code to support that hardware in U-Boot is about one year old. So this one was only updated, see above.

The Palm LifeDrive is a different thing though. Because I wanted to avoid destroying my unit, I figured out a way how to pseudo-dual-boot PalmOS and U-Boot for testing purposes. This turned out to be quite simple, though without JTAG one has to be very careful. Firstly, the size of flash in Palm LifeDrive is only 512kb -- 0x80000 bytes -- and the last 0x20000 bytes are unused and PalmOS ignores them. That's enough for slightly stripped down U-Boot binary, but there's one more thing missing, that is, in case U-Boot was installed in that part of flash, something must execute it.

Executing it means jumping to it's first instruction. By looking at the disassembly dump of the Palm LifeDrive flash, very close to the beginning, the code clears register "r0" by moving zero into it. By replacing this instruction with a jump at the 0x60000 address, the U-Boot can be executed. But that would make the dual-boot impossible in case there was just pure U-Boot at 0x60000.

The trick is to add code to 0x60000, that checks PSPR register aka. the only register that survives suspend-to-mem. This register always contains value that modulo 4 gives no remainder, ie. is aligned to four bytes in PalmOS. The last two bits are unused though and that's the trick.

The point is to load U-Boot just after this code that checks PSPR. In case these last two bits in PSPR are set, the code will not jump right past the place where the "r0" was zeroed, but instead will just continue executing the code further on, which means CPU will reach U-Boot. Otherwise, the PalmOS loader will just continue execution. I also had to zero the "r0" before jumping back to the PalmOS loader in the later case.

One more thing is important here. To set the PSPR, I adjusted the suspend routine in Linux kernel to store some totally broken number into it and when I then pressed any of the "unsuspend" buttons, the U-Boot came up. Note, the U-Boot I used for this had the wakeup capability disabled, which is very important.

To get the U-Boot wakeup the kernel properly, it's necessary to replace the Palm LifeDrive bootloader altogether. That's what I did and it worked very well for me, until I wanted to test a newly compiled version for Tomas Cech. The battery choked while the system was being updated. Well, that's sad. Luckily, I already have a replacement device.

Cypress EZUSB SX2
Having U-Boot on Palm LifeDrive means it can now be fully used with Linux. That also means need for some kind of connection to the outside world. Most of the people involved with Hack&Dev certainly recall the USB 2.0 chip in Palm LifeDrive. The chip is actually quite simple, but also quite incompatible with Linux's perception of an USB gadget chip.

The SX2 requires the so called Descriptor RAM to be filled with proper USB descriptor in order to enumerate at all. I started writing a driver for the SX2, but it's still very incomplete. In the current state, it can send data from host and to host and is behaving as 4 bulk endpoints. That implies it can be used probably only with g_serial for now. But it's still far from complete. The draft can be found in my kernel tree and anyone willing to hack on it is welcome.

The drinking party
During the Linaro conference in Prague, me and some other people -- Eric Miao, Nicolas Pitre, Daniel Mack, to name a few -- went for a beer. The party turned out into an awesome thing, it was probably the most enjoyable evening and night of the year for me.

Anyway, Eric decided to give up his maintainership of the Sharp Akita and pushed it into me. The current support for Sharp Akita in mainline is kind of poor, but with improving tendencies. The power management code is very broken and there are still a few other issues.

I decided to try fixing the power management issue so I joined efforts with Pavel Machek who started putting together a new battery driver for Sharp Spitz. Actually this also work on Sharp Akita and Sharp Borzoi. The driver currently has more rough edges than I'd love it to have, but it's kind of usable given enough knowledge on the user side. The problem with it is, that it still doesn't check the battery temperature so the user has to be careful not to overcharge the battery. Actually, there should be some GPIO, that prevents the battery from turning the device into a lethal weapon and the driver checks this GPIO, but according to Pavel, this is not safe enough.

I met with Wookey the other day and got a Balloon3 board with some additional peripherals. That obviously ended up in me porting U-Boot to this board and doing some heavy rework on the side of mainline kernel.

The first problem I faced was the FPGA that's on the Balloon3 board. That's used to route various CPU lines to hardware. It also controls NAND, CF and such. The PCMCIA/CF driver had to be slightly fixed on the Balloon3, because in case of FPGA, the only working firmware is version 0.3. For CPLD, which is integrated on some other Balloon3 boards, there is already version 1.0 of the firmware which has a different behaviour. I hope the fix for the FPGA firmware is underway as Hector Oron told me he'd look into it.

The worse thing was the NAND driver. For that, I used the gen_nand driver and implemented various necessary callbacks based on the original out-of-the-tree NAND driver for Balloon3. I heavily reworked this so the driver was used only as a technical documentation. Pretty much all the hardware is implemented on the board I have by now.

When I was hacking on the NAND driver, I noticed a bug in the gen_nand driver where it ignored the number of chips passed to the driver. The gen_nand assumed the number of chips was always one, which was true for all the boards that used the driver so far. But on balloon3, there were four chips in total. I fixed this and even implemented a check for invalid value of number of chips as per David Woodhouse's suggestion.

Toradex Colibri PXA3xx
Recently, there were a few people concerned about the status of Toradex Colibri PXA3xx. I was aware of a fact, that the code needs to be reworked for quite some time already. Just recently, I was motivated enough to go forth and do it. Therefore the code that's now in my tree is the reworked support for the board and the CPU card. I also reworked support for Colibri PXA300/PXA310 while at it. It is done in a very similar fashion to the PXA27x Colibri support. Actually, I used the evalboard code for Colibri PXA27x. Now all the Colibris use the same evalboard code and that's how it should be.

This is not all though. There was some stuff going on in the OpenPXA project about the Colibri PXA320 recently too and I was also pestering the tech support in Toradex. The result is that the E-Boot bootloader supplied by Toradex is not the only one that can update the CPLD firmware on the Colibri PXA320 board anymore.

Toradex provided me with the information on how the CPLD JTAG is connected to the CPU GPIO pins and an xsvf file for the CPLD containing the latest version of the CPLD firmware. And by using an xsvf player that I found is already in U-Boot and used on some PowerPC platforms, I was able to reprogram the CPLD to the latest version v1.7 of the CPLD firmware.

The xsvf player needed some obvious changes to operate the GPIO-emulated JTAG correctly, but the code is well abstracted and I think I'll make it into a generic code soon. Currently there are two copies of the same code in my U-Boot tree, which is no good. Note that the v1.7 of the CPLD firmware is probably some internal testing version from Toradex and I still have to ask them about the license for that file. It'd be best of they just put this xsvf file on their website.

Still, this is not the end. Because I wanted to use CF memory card on the Toradex PXA320 board, I decided to start implementing the PXA320 PCMCIA support. I first had to patch the pxa2xx_base to support different locations of the Static Memory controller registers, the MCIO, MCATT, MCMEM and MECR registers. That was the easier part.

After a bit of struggling with the Toradex CPLD and GPIO pin configuration and muxing, which is doing some obscure stuff to the PCMCIA signals, I managed to get the PCMCIA partly working. Note the struggle means I was stuck with it for more than a two days until I figured out what Toradex actually did. Their tech support on the other hand is very good and helpful even if a little bit slow, I'm happy about this.

The partly working above means, that if I insert a Marvell CF8385 into the CF slot, the card is detected, firmware is downloaded and no problems observed. Once I insert a memory card though, nothing happens. The card insertion GPIO is asserted, but it probably reads zeroes or something from the CIS. This issue can actually be just about anything, starting from wrong timing to misconfigured GPIOs. The more confusing thing here is that if I insert an Emtec or Patriot card, the card insertion is only detected. In case I insert a SanDisk card, the pata_pcmcia driver is loaded, but in the end, the disk is not recognized.

The cluster
You might have noticed the hardware kind of started piling up. That's true, but it's not unused actually. I use the boards that are in good shape software- and hardware-wise as an ARM compile cluster nodes. I run distcc on these and use it to build Debian packages. It's also an awesome stability testing and bug hunting mechanism for these platforms.

The cluster is still growing, but currently there are these machines:
  • Palm Tungsten|C
    • PXA255 400MHz
    • 64MB RAM
    • 1GB MMC+ card
    • USB CDC Ethernet
    • Debian SID

  • Voipac PXA270
    • PXA270C5 520MHz
    • 128MB RAM
    • 2GB SD card
    • Ethernet connection
    • Debian SID

  • Voipac PXA270
    • PXA270C5 520MHz
    • 256MB RAM
    • 160GB harddrive
    • Ethernet connection
    • Debian SID

  • Balloon3
    • PXA270C0 520MHz
    • 384MB RAM
    • 1GB NAND + 4GB CF card
    • USB Ethernet connection
    • Debian SID
More devices will be added as they will reach good enough state. I was also thinking of granting access to the cluster to a few fellow developers once it stabilizes well enough.

Other
Just a few days ago, Mike Rapoport aranged it so I received a CM-X300 board. I was still unable to replace the bootloader there as I still don't have JTAG connected to that board.

I also started plumbing into the Debian u-boot package and prepared a patch. The problem is the current maintainer is totally ignoring any requests from me so far. I hope it's just temporary as he was busy with organising the debconf.

I hope you enjoyed reading the text.

No comments: