Wednesday, February 24, 2016

Tweeting from your Pi - Fixing issues with Tweepy

Edit (10/21/2016) - Debian Jesse took care of most of these problems, so you may not need to do all of them. I ran through on a fresh Jesse install just to see what happened, no errors but there was a lot of "already installed" messages returned. YMMV!

I've set up one of my Raspberry Pis to do some various functions around the house - monitoring a weather radio, checking temperatures - that sort of thing. I have it send alerts via email, but wanted a more publicly accessible way of getting this data in the event that I don't have my phone with me, or whatever. You know - more redundancy is better!

Twitter is a pretty good way to satisfy that requirement, and they provide a pretty decently documented (and well hashed by programmers) API. It's pretty easy to push messages from the command line with a few lines of python.

I started by following this guide to create the application in twitter and get the necessary python libraries on my Pi. It's using Tweepy, which seems to be the de-facto way of sending data to twitter via python. For the most part, the guide works, but I did run into some issues.

First was this error while actually installing the Tweepy library:

TypeError: parse_requirements() got an unexpected keyword argument 'session'

Searching for information, it seems that pip needs to be updated. Now, you're saying to yourself, "I just installed this! Why does it need to be updated?"

The version in the repository is an old version, which is the reason for the needed upgrade. See this post on github for a description of the problem. It's easy enough to correct, just run the upgrade command on pip:

sudo pip install --upgrade pip

Great. You're now ready to tweet. You've put your keys in the programs from the guide, you execute the program, and:

/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:315: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:120: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning


Well, crap. The tweet worked, but this warning pops up. Certain cryptographic extensions aren't available on the Pi. Easy enough to fix, see this stackoverflow question for more information.

To fix THIS issue, you'll need to install some more stuff. I assume you already have a gcc compiler installed, along with dev tools for it. There are plenty of tutorials out there for that, so again - I'm not going to cover it here. However, once you have that set up, you'll need to install some libraries, and then install the cryptographic extensions:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install libffi-dev
sudo apt-get install python-dev

What that just did is install the Foreign Function Interface that allows Python to use compiled code from other languages. This actually has to compile some code, so it will take a few minutes (or longer, depending on your Pi.) It also installs some python development tools so gcc can properly compile python support into the next command.

Now:

sudo pip install requests[security]

That will install all the necessary security extensions. Again, this will take a few minutes. If you get other errors at this point, I would suggest just copying the error directly into a google search. Chances are, someone else has had the same error - in fact, that's how I solved all the problems here.

Now, you can tweet with Tweepy. You shouldn't get any kind of messages on the Pi unless the tweet failed in some way.

This is a fairly top-level overview of what I encountered when installing Tweepy. As always, if you're unsure of what the steps mean, or encounter other errors during the installation process, STOP! Research your particular error before continuing.

Tuesday, February 23, 2016

Replacing a commercial NTP server with a Raspberry Pi

This past week, my rack-mount, commercial GPS NTP box suffered a failure. Apparently, the GPS receiver was unable to make sense of an extended week count number, rolled over, and now thinks it's 1996. There's a firmware patch for it, but mine didn't seem to like it (the manufacturer has been very helpful and we are working through it.)

While I'd certainly like to think it's now 1996 again (let's see, buy some silver, some Bitcoin, sell Lucent stock...) it's not, and having your network clock so far off isn't the best thing for time-sensitive signed certificates.

In steps the Raspberry Pi. While there have been a number of ways to use a Pi as a NTP server, such as this guide, the hardware they rely on can sometimes be difficult to get, especially if a few years have passed since the article was written.

That guide did show a link to a piece of hardware that is still available in an updated form.

This board is designed to drop directly on top of a Raspberry Pi + or 2, and provides, according to the vendor, an adequate enough service for a Pi-based NTP server, as well as stationary mode and PPS output. I take that to mean that it's not going to be as good as say, a US$5000 unit, but is going to be much better than trying to sync from internet-based time clocks, especially if you want all of your local devices to be very close in reported time. They even provide a helpful how-to guide for setting up a stratum-1 NTP server.

You can order one of these boards from habsupplies' online store. I currently have a board + antenna on order, when I can get it installed I'll do a quick write-up of my experiences.

Note: The original problem was older Trimble GPS units rolled over at week 1024, taking the date back to 1996. The firmware patch corrects this by, I assume, making the attached computer think it's year+20. The original device is now working again - I had to get special instructions from the manufacturer of the NTP box regarding disabling certain watchdogs to allow the firmware update to complete.

Monday, February 15, 2016

Unusually high load average times on my Raspberry Pi?

I run a few Raspberry Pi 2 on my network. They handle basic tasks - network and device monitoring via Cacti and Smokeping, a local email and chat server, and an ad-blocking DNS server, ala a Pi-Hole. None of these services really task any of their respective machines terribly, so when load times on the Cacti/Smokeping machine started creeping up past a 1m average of 1.06 from a 1m of 0.05, I got a little concerned. It didn't seem to be affecting the machine adversely, and top didn't show me anything unusual, so I let it go as it seemed stable.

That is, until a bootloader update came along recently. Attempting to install it threw I/O errors while the installer tried to copy the new files to /boot.

 I boot my devices from an NFS share that sits on the network, but due to the way the Pi is designed, you still need a single FAT32 partition on an SD card that contains the bootloader's binary blob. This partition gets mounted as /boot at runtime, and is the only partition residing on the SD card in my particular case. Everything else sits on the network share, safe from the corrupting influences of the SD card.

Attempting to ls the /boot directory revealed an empty directory!

I was really happy I hadn't tried to reboot the machine remotely at this point. I shut it down and pulled the card. While the files normally there were still there, the directory was considered dirty by Windows and wanted to be checked on mounting. I was, however, able to copy files from the /boot directory onto a new card, which consisted of a single FAT32 partition.

I reinserted the card, and the Pi rebooted normally at this point. Updates installed, everyone cheered, and I forgot about it until today. Logging in, I noticed that the load averages were now back to what I considered normal for the light load on the machine! I'm not sure why having a bad /boot directory caused my load times to skyrocket, but the cause and effect seems to be there. The machine is running happy again, and my original thought of having the actual OS offloaded in case of SD card failure proved to be a sane one. The whole operation took about 10 minutes to correct, most of that time trying to find an SD card that I could erase.

I've had problems with SD cards simply failing, but that was with the entire OS on the card. Having only part of the system on the card allowed it to continue running, just...strangely.


The happy Pi, chirping once more...

 I'm sure there's an explanation for why the load times were high with a bad boot sector, but I'm unable to explain it. Perhaps you can?

Saturday, February 6, 2016

Burroughs B-7971 Nixie Tubes...

A long time ago I acquired some of these giant Nixie tubes from some old equipment. I had intended on doing "something" with them, but never got around to it as the sockets for the tubes were as expensive as the tubes, and I didn't really have enough to do a whole lot with. I didn't want to use them in a clock, but that seems to be the best use for them.

These are fully segmented alphanumeric tubes, and are about the size of a 12oz can of soda!


I honestly can't remember what they came out of, other than it was some big industrial controller. They were working when pulled, and I don't have any reason to think they are not operational right now. Considering these things date to 1964 and 1967, that's pretty impressive.

They're really dirty, but I hate to clean them as it may destroy the printing.

They're all the same model, but they are all marked a bit different. 5 of the tubes are marked with the Burroughs "B" but are labeled "Ultrasonic Systems Corp." - I can't find any reference to Burroughs and this other company, so I have to assume they are a OEM marked part for this other company.


These guys were all dated to around 1967.

The other 3 were simply labeled as Burroughs, and dated around 1964.


The best thing to do with these is sell them off. If you're interested, please let me know and I'll direct you to the craigslist ad that they are posted under. I prefer local pickup, but will ship (postage, insurance extra) to most places.

Now, all together for a family portait (I see you hiding in the back!)


Hopefully these guys will find a home. 

B7971 and B-7971

Edit: As of 2/29/2016, these tubes are SOLD. Thank you.

Monday, February 1, 2016

A small rack system for a friend.

A friend of mine wanted to set up a small home network rack in his basement, but needed to do it as inexpensively as possible. We already had most of the normal equipment you'd set up in a home system - the modem provided by the cable company, an older (but still operational) Linksys WRT-54GL router and WiFi access point, and some other odds and ends.

In order to get it all set up, I needed a small rack. Craigslist was the first choice for this type of item, and browsing revealed a number of different items available. I settled on a Cooper half-rack on wheels, which I picked up for $50 (It fit in my car.) It appeared to have had some sort of high-speed fiber termination in it, as there were a couple of fiber patchpanels, and some Cat6e cables hanging out as well. A cable management rack was included as well, but the plastic had been shattered and it was discarded. I kept as much of the extras as I could, which was mostly just the Cat6e cable. These were about 1' long, and I set them aside for later. Pretty much everything else in the rack was discarded, as it was chopped or broken.

In order to add some good functionality to the rack, I located a seller with a slightly used rack-mount UPS (in a nice carpeted rack box) for another $50. Another Craigslist find, the seller was moving and wanted it gone. The PDU above it was a demo from some local company that had dead LEDs in the display - these were easily replaced. An AVTech RoomAlert 32e, left over from another job where I was able to purchase 3 of these units for under $50 was mounted as well, with some of the homemade sensors I described earlier. A discarded D-Link 10/100/1000 switch that was recovered from a junk pile at work after a switch upgrade was bolted in. Shelves were purchased surplus. Full of holes, but who cares? A Monoprice patch panel and some keystones rounded out the project.

We made some new cables with new wire and ends to run through the house, but I also reused all of the Cat6e cables that came with the rack. These turned into jumpers between the patch panel and the switch.

The rest of the equipment in the rack was already owned. Two Synology units and the aforementioned modem and router. As payment for setting things up, I requested co-lo on my own Synology unit, which along with it's accompanying APC UPS, makes a nice off premises backup system.

It's not the prettiest thing in the world, but it keeps everything in one place, and allows for airflow and offers a little bit of expansion if needed.


In all, the project totals, for things we didn't already have:

Rack and Jumpers: $50
Patch Panel: $20
Keystones: $20
Switch: Free
Room Alert device: $20
PDU: $65
UPS: $50
Shelves: $20
Cables and misc: $100 (we didn't use all the cable for this job.)

TL;DR: When setting up a small system, Craigslist and eBay can be your friend.

Bitcoin?    1B1DWLzvgp3tdtNjbHufyqRP1fUdN6HXLh