25 October 2016

Oblogs may be abandoned soon (info post)

If you've found something of value in this blog then I'm happy to have helped you. It is the secondary purpose of why I created it, with the primary being to remind me of some of the technical tasks I've been faced with and a way to document my solutions. I am blown away by the fact that over 6,600 of you have viewed my post about HTTP posting using App Inventor. Unfortunately I am now facing (yet another) restructure at work and if I am forced to leave the education sector, I will have more pressing concerns (and possibly a few less technical requests to deal with).

Ironically I might end up with more time on my hands so the direction of the blog could change if I continue with it. If so I will probably focus a lot more on Linux (certainly Raspbian), Python and possibly C++ (which I've always wanted to learn but never really had the time). I am thankful to the people I work for who have taken me away from operations and technical fixes to some extent and pushed me towards front-end web development over the last two years. I now have a pretty good understanding of Javascript, restful API's and json.

I am also quite excited about having time to go back over a project which was abandoned that I thought was a little bit special. I will be looking over my prototype backups and starting again with a total rewrite of my original idea. Free time and freedom to release it as an open source project will allow me to focus my efforts on the parts which I know can be improved. I might also be able to use any new skills I develop to create the tools which don't currently exist which we needed to support the project and are ultimately the reason why it was dropped prematurely. Next year could be a very exciting time and despite the chaotic rush of all around me trying to figure out where they fit into the latest rehash of our technical structure, I'm quite optimistic about my future prospects which is definitely a good thing for someone who is normally quite pessimistic and may have been held back in the past for playing devils advocate once too often. It's a shame #Techxit has already been used as it's quite appropriate to my workplace right now.

To infinity, and beyond...

10 June 2016

BBC Micro:bit - Downloading accelerometer data to a linux PC

If like me, you've finally managed to get hold of a BBC Micro:bit and want to try out the accelerometer data-logging function (as used in the Bloodhound competition), you might find it more difficult if you're using linux.

Well if you're using PC Linux OS, we can fix this.

First of all, use the Micro:bit website to program the device. I struggled to find the data-logging parts so I recommend you actually use the tutorial for this part.

Now for the hurdle. The tutorials I've seen for downloading the data suggest you use an old Windows-only terminal emulator program called Tera-Term. This is quite unique as it lets you copy the data in table format. I couldn't get it to work on linux (using wine) and found the micro:bit serial instructions for using linux to be very poor (and untested). My version presented here uses current software and HAS BEEN TESTED (at least on PC Linux OS 32-bit mate edition running on a Samsung NC10).

http://www.linuxquestions.org/questions/linux-newbie-8/screen-terminates-immediately-4175444530/
Was very useful for creating this fix

Go to synaptic package manager and install screen (terminal emulator)

Start a regular terminal
List connected ports using

dmesg | grep tty

should show... ttyACM0 : USB ACM device (this means the microbit is attached via port /dev/ttyACM0)

To connect to the microbit using screen, we therefore use the command:
screen /dev/ttyACM0 115200

which will most probably result in an error like this one...
error /home/*username*/tmp must have mode 700

To fix this:
mkdir -p /home/*username*/.screen
chmod 700 /home/*username*/.screen
cd /usr/bin
su root (enter password when prompted)
chmod u+s screen
chmod 700 /home/*username*/tmp
exit (superuser mode)

I then tried the screen command again and pressing the right button on the micro:bit actually downloaded the data to the terminal. From there you can use the edit -> select all and edit -> copy functions to grab the data for pasting into an empty text file or office document. I also found it useful to go back to the micro:bit code and add some additional images before and after the micro:bit -> export command just so you can tell when it is attempting to dowload.

I would also like to point out that I am not a linux expert and this fix was performed on a laptop which is not usually connected to the internet. Any use of this fix is at your own risk so go search that chmod command and try to figure out what you're actually changing.

The good thing about the micro:bit site is you can download your source code and also the (.hex) executables which are simply dragged-and-dropped onto the micro:bit. So once you have these, there's no need to go back to the code editor unless you want to change things. Internet access is only needed for installing the screen package which is why I'm not too worried about the permissions changes on my laptop. Please consult a linux expert if protecting your system is a higher priority.

Linux on the Samsung NC10

My trusty old NC10 still remains one of the best devices I've ever bought. Linux Mint gave it a fresh lease of life but as I've mentioned before, the problem with the 'Disks' utility is an issue for me. Mint is an easy sell for RPi enthusiasts as 'Disks' allows for easy creation of SD-card images, however this feature is still not working on my netbook (v17.3 32-bit). I have searched for answers and found various hints at where the problem lies, such as here and here but sadly no solutions. I did try removing 'Disks' from mint and then re-installing but the result was a perfectly working 'Disks' utility until I rebooted - at which point I could not login again. My final conclusion is that the bug only exists in the 32-bit versions as I have no issues on the 64-bit desktop versions I have installed on other computers. Unfortunately I the 64-bit version doesn't run on my old 2008/9 NC10 with a 1.6Ghz cpu. The 32-bit version seems quite happy to image my 2GB Riscos image but fails with my 8gb+ Raspbian images suggesting it's a filesize issue (cause by the 32-bit version of lhseek?)

So I've been trying out some lightweight Linux alternatives as suggested here. All I actually wanted was a working OS which detects the hardware ok and if possible a working version of the Disks utility (also known as gnome-disk-utility). If it also included Gparted, thunderbird & audacity that would be a bonus but since the Pi projects are a lot of my work (& life), my main requirement is a working image creator utility

When I looked into a good distro to try, the list I found via Google looks very out-of-date (Nov 2008) so I was quite looking forward to some fun experiments. What follows is my oppinions on what I found. Before anyone takes offence, all of the people working on these projects deserve a lot of respect for the time and dedication they have obviously put in to their projects.

1. Absolute Linux
My first introduction to Linux came from a guy who was always a big fan of Slackware Linux. I dabbled a bit in years gone by with original Slackware and some derivatives (Puppy/Slacko). They were great but I found Slackware could be a nightmare if things didn't 'just work' (to quote Apple fans worldwide). Most things were fixable but could be a lot of additional effort for simple things like getting a mouse or sound to work. I was surprised that the installer still looked like it belonged to the 1990's. Never judge a book by it's cover though. In true Slackware style, the distro installs packages flashing up info about them in it's retro dos-style window. Some flash by before you can read them while others remain on-screen for a while as they are uncompressed and installed. I did not really expect to see an iOS app installer in the package list so that alone got my curiosity going but overall, installation was frustratingly slow. I actually left this one until last as I really wanted to like it but sadly it could not become my new favourite distro. On the plus side, it picked up my hardware without any issues (the mouse pad and networking worked without any issues which was great to see). Overall, Absolute was not as polished as some of the other distro's in the list and I really did not like the fact that packages I'm already familiar with had been renamed to be more 'newbie friendly' (for example the 'Drive info & formatting' option starts up Gparted and the 'Audio Editor' likewise starts Audacity)

2. Trisquel-mini
From the article, Trisquel appears to have a very similar application manager to mint so I was hopeful. Again the mousepad & wi-fi worked from the very start. The user-interface is incredibly clean and really easy-going on the eyes. I did not make a note of how I found the gnome-disk-utility, but I did get it loaded. Sadly it suffered from the same quark-13 error. I'm not too surprised by this as I suspect what I'm really hoping to find is a distro with a repository which has addressed the issue within the source code. I could see myself happily using Trisquel if time allowed me to make a few more changes to the installation (given a choice of Midori or Firefox I would opt for Firefox every time).

3. Tiny Core
I haven't tried Tiny Core as yet because I am planning to try that along with it's Raspberry Pi version together. I will post my findings after I've had time to do this.

4. Lubuntu
Alas poor Lubuntu. This may be another distro to come back to as the fixes I've seen for the Disks issue were found on Ubuntu forums. The downside of having 'Lube' on my netbook was avoided sadly because it never got further than a blank screen with a flashing carat in the top left corner.

5. LXLE
LXLE is a very pretty distro, and like many of the others tested here, it picked up the hardware and even the wi-fi after some tinkering to connect to the right hotspot. If I was looking for a pretty OS rather than a specific need, I would be tempted to revisit this one. Out-of-the-box, it has Seamonkey which I would want to replace with Firefox asap. Again the gnome-disk-utility was not installed as standard but I figured out how to install it quickly enough. Sadly whichever repository it came from, it still had the quark bug.

6. MX Linux
As with LXLE, this is quite a pretty distro although I was not overly keen on the side menu bar (which I know is just a preference and I'm sure it can be changed). Again the hardware was all detected and seemed to work well. Wi-fi was a doddle to get going - just pick the access point and give it the password. To it's credit it also had Firefox and Thunderbird installed. I didn't think the start menu's were as clean as other distro's though (LXLE & Trisquel). Again the gnome-disk-utility was not installed as standard but installation was a breeze. Sadly once again the component I need most gave me back the now all-too-familiar quark-13 error.

7. Porteus
Porteus like Absolute is based on Slackware but without the lengthy install. This is acheived by having you select things about your OS from the website when you download it. Know which window manager you want to use? Then that's what you get in your image file. It's an idea that seems to work and my 'Mate' image seemed quite happy with the NC10's hardware. On the plus side it also had Firefox installed so that was another point in its favour. Unfortunately this download approach also has its disadvantages too. There was occasional screen flickering on my NC10 and the gnome-disk-utility was nowhere to be found. The package manager also seemed a bit archaic and when I went hunting for g-d-u I was presented with a nice red 'Missing DB files' message.

8. Slitaz
Slitaz... it has it's very own distinct feel to it. Administration tasks like installing extra packages are done using the TazPanel - a sort of HTML approach, not unlike Microsofts recent Windows 10 developments. Again mouse-pad and wi-fi were picked up automatically and it seems to run well on the NC10 and has a very stable look and feel to it. The pre-installed apps are not really my choice (Midori again, although Firefox was simple to install). I couldn't find the gnome-disk-utility in the package manager, even though the full gnome desktop seemed to be there. Again the interface is clean and usable. I wouldn't be surprised if I end up installing Slitaz on another system at some point in the future. For now though it doesn't quite fit my needs.

9. Vector Linux
Vector linux has a very cool boot-up screen and a look and feel that makes me think it's another Slackware variant. Maybe that's just what the Ice window manager reminds me of although the gslapt package manager also has that Slack feel to it. It was good to see Geany installed as standard and like many of the other distros, the NC10 hardware seems well supported. Mouse-pad and wi-fi were ready at boot-up. Once I found and updated the package manager, it was easy enough to find & install the g-d-u. Unfortunately it turned out to be version 2.32.1 from 2009 which doesn't appear to have the imaging options :(

10. Puppy Linux
As I mentioned previously, I have used puppy before. I remember it being a very simplistic distro and very, very lightweight and fast. I ran it from live disks and it was one of the first versions I ever got to run from a USB drive. I even had an older version running on my NC10 at one time so it's like an old friend I haven't seen for a while. I also have a version which runs on the Pi in my SD card collection. The version I tested is the Tahrpup 6.0CE non-pae version. I was optimistic as I read somewhere it uses the Ubuntu trusty repositories which I think include the g-d-u fix. It was sadly dissapointing to find this pup did not support the mouse-pad, sound or wi-fi from its set-up disc

Well given my history of using this distro, I decided it needed a second chance, especially after Lubuntu had also failed to work for me. I downloaded Puppy Slacko 6.0.3 as despite the issues with Tahrpup you can see the effort the puppy group put into making things easy to set-up. I'm happy to report that on boot-up, the mouse pad worked, as did sound (which I discovered when my NC10 suddenly barked twice at me). I was then prompted to set-up the network connection and the simple set-up script which had failed to find any devices in Tahrpup showed wlan0 was there and it connected just fine. Despite it's dimminutive size, Slack includes Firefox which was a nice discovery. A quick search proved the wi-fi was indeed active. I opened up the puppy package manager and updated the package databases and went looking for the gnome-disk-utility but I couldn't find it. I eventually found a listing for gdm and had one of those sudden moments of enlightenment. The version listed was 2.20.11 so I set about installing it although suspecting it would turn out not to have the image creation tools. I still don't know where it installed or how to run it.

While scouring the menus though I found something called 'Pudd'. It looks like I'm not the only person looking to simplify SD card duplication as Pudd is the Puppy Universal DD tool - essentially a script which prompts you for a source (e.g. usb SD adapter or SD card slot) and a destination (e.g. file.img). Unfortunately this uses DD rather than DCFLDD or DC3DD which give feedback during the cloning process so once the Puppy script is going it could be a long wait without any feedback (my own tests with DC3DD using defaults resulted in almost a whole hour to clone an 8GB SD card - a task which g-d-u can accomplish in around 20 minutes on 64-bit linux mint). Pudd also compresses the image too but I'm not yet sure how it will react to the age old problem of cloning an image to a card that is 5-125mb smaller than the original card. A workaround for this is to use Gparted on the original to resize the partition leaving a block of unused space at the end of the card. DD fails when the end of the smaller card is reached but by that point it has written the completed partition so it fails while copying empty blocks resulting in a copy which is still usable.

So after a couple of days testing I can now confirm that my new favourite distro for my NC10 is .... PC Linux OS 32-bit community edition

Ok, a distro not on the list but after so many fails, the problem required a different approach. Instead of randomly testing distro after distro, I decided to see if there was a way to check which distro had the most recent version of g-d-u installed. It turns out you can check this using RPM search (at least for packages bundled into an RPM for easy installation). There were a few familiar distros there but one I didn't recognise had g-d-u version 3.6 mentioned within a 32-bit build so I opted to try out the 32-bit mate edition of PC Linux OS v12.15 (community edition)

PCLO (ELLO ELLO...) looks quite similar to Porteus and picked up the mouse-pad and wi-fi automatically. It's also very colourful (as is the new g-d-u installation). I even learned where I might be going wrong with some of the other versions. In Linux Mint 64-bit, clicking the drive and selecting create image seems to automatically unmount or ignore any connected file-systems on the card before it starts imaging. It's this sort of attention to detail which I look for in a distribution, but then you think it's a feature of the program and expect the same on every other distro. The more colourful and different interface of PCLinuxOS got me wondering about the tool as on my first imaging attempt I got a different error (quark-0). I also noticed a slight difference to the gui below the card structure diagram (the bit that looks like a Gparted view). Right-clicking gave me the option to unmount the cards file system (which had been auto-mounted opening up the files utility in the process). I unmounted each partition which had the option to be unmounted and the icon below the card structure changed to a right-facing arrow (>). I tried again. SUCCESS!!!! It was happily creating a copy of my SD card and even better - it would only take around 8 minutes (essentially 1Gb/minute).

G-d-u still fails to restore an image back to a slightly smaller card but I'm hoping I can install DC3DD to cover this scenario. The fact I now have mobile imaging on my netbook is a major success and has me wondering if maybe not knowing to unmount file-systems prior to starting the image caused some of the problems on other distro's. I actually don't think this is the case as the error (g-io-error-quark, 13) has quite a specific error message "Error allocating space for disk image file: Invalid argument". so I'm fairly sure they suffered from the same issue as Mint and need to upgrade to a newer version of the tool. If any of the g-d-u devs find this page, it may be worth tweaking the code to unmount those file-systems before the process starts as it's not obvous to us lesser mortals (a.k.a. users). I pretty much found it by accident.

To conclude, all of the distributions I've looked at are praiseworthy in their own way. I am merely an end-user looking for a tool to assist me with a specific use. It's quite a good test using specific hardware as in theory anyone else using a Samsung NC10 should be able to replicate my testing. These distros try to provide an operating system that works on many platforms (like Windows) but without the money or developer teams that M$ has. It's great to see how much progress has been made with these versions compared to the distros I tried out over a decade ago. To see the hardware supported out-of-the-box in the majority of cases is superb.

I will be spending some time this weekend putting PCLO through some more testing and configuring it to my liking. Well done and thanks to everyone in the PC Linux OS developer community.

UPDATE

Shortly after all this testing, the RPi foundation announced a new Raspbian version which now has the ability to clone the system SD card to a blank card (connected via a USB adapter).

9 March 2016

Pi sense-hat slideshow

Just recently I've been looking into using the Pi sense hat to control screen content. What I expected to be quite simple turned out to be harder than expected for one reason or another. What I set out to acheive is a picture slide-show which could be triggered by moving the Pi (with the sense-hat attached).

The first issue I encountered was using PIL (the python image library). It seemed to just be a way of passing filenames from python to a seperate image handler within Raspbian. PIL seemed to pass the data to either 'xv' (xviewer?) or 'display' which you can manually trigger from a console using 'display filename.png' or 'xv filename.png'. I encountered articles that suggested installing something called 'Image Magick' if this didn't work. Fortunately it did work, but not how I wanted. From within python, it would start a new instance each time I tried to open it with a new filename so instead of a single window with a picture that changed, I ended up with multiple pictures which also had an icon of the image slapped right in the centre of the image. Not particularly useful for what I wanted.

For my next tests, I though I'd try the same thing in Scratch as I read Scratch is able to input/output from/to the sense hat by referencing the GPIO. Unfortunately the accelerometer sensor support isn't quite there yet. I experienced frequent crashes, especially when trying to make Scratch play nicely with the sense-hat. I went back to the drawing board.

I had to go back to using python but this eventually paid off once I'd done a lot of searching and experimenting and installed a few extra applications (and changed the way my code works). First of all I went through a succession of different image viewers and finally found the right one for me - called 'Feh'. I also changed the way I send images to it. Initially I was trying to cycle through a list by continually changing the filename using python strings. In the final version, I give Feh the list of files to use and then I check if the accelerometer readings show any signs of movement. If so, I then emulate a user pressing the 'advance to next image' button in Feh. Doing this requires another installation (known as x-automation).

The end result is not perfect as it shows errors on the console when you quit 'feh' (using either Q or the escape key), but it does sort of do what I wanted. The instructions and code below are a memory aid as I know I will probably need something like this again at some point.

Set-up (Pre-Requisites): From the command line, enter:

sudo apt-get install feh
sudo apt-get install xautomation

Create a folder on the desktop called python and a subfolder in there called images. Save .PNG images into this folder, numbering them sequentially (e.g. 1.png, 2.png to 8.png). Create a new file in the images folder called slideshow.py (You can use the Geany editor for this. Use sudo apt-get install geany if it's not already installed). Just copy the code here and save it.

 from subprocess import Popen, PIPE  
 from time import *  
 from sense_hat import SenseHat  
 sense = SenseHat()  
 pic = 1  
 #cursRight = 'keydown Right'  
 #cursRightUp = 'keyup Right'  
 kN = 'key N'  
 def keypress(sequence):  
   p = Popen(['xte'], stdin=PIPE)  
   p.communicate(input=sequence)  
 proc = Popen (["feh", "-x", "-F", "-N", "-Y", "1.png", "2.png", "3.png", "4.png", "5.png", "6.png", "7.png", "8.png"], stdin=PIPE)  
 # cmdline args enclosed in quotes, seperated by commas - all enclosed within square brackets  
 while True:  
   x, y, z = sense.get_accelerometer_raw().values()  
   x=round(x, 0)  
   y=round(y, 0)  
   z=round(z, 0)  
   m=x+y+z  
   # m is the sum of all three axis   
  # z seems to typically have a value of 1 so this becomes our baseline for movement detection  
   if m>1 :  
   # i.e. something moved  
     pic = pic + 1  
     if pic>8 :  
       pic=1  
     #we could use Popen here if we wanted to run different shell commands e.g.  
     #fname = "/home/pi/Desktop/python/jd/" + str(pic) + ".png"  
     #flop = Popen (["xv", fname])  
     keypress(kN)  
     #kN is defined above. This passes 'key N' to the keypress function which in turn outputs it  
     # to 'xte' using its stdin pipe (via p.communicate)  
     sleep(1)  


To run the program, you need to be running in the x-windows environment (startx if your pi boots into command line). Start a command terminal and enter

cd Desktop/python/images
python slideshow.py

Sit back, give your pi (with the sense-hat fitted) a slight shake and it should cycle through the images in response to the movement. Feel free to expand on this code or create a better version. The additional settings after "feh" set a few command line switches for full-screen and turning off comments etc. Refer to the feh docs for info on these (or use the command line feh --help). Code on this page was formatted using CodeFormatter - thanks to the author of that little gem whoever you are :)