When Atomic Clocks Stop Ticking!

Back in December last year I built my GPS governed NTP server and it’s been working perfectly. I never managed to get the clock display to an external TV screen project working simply because I couldn’t get a signal out of the various composite cables I had laying around and shelved the project at the point of having a competent and functioning reference clock for all my other machines.

Everything has been fine and dandy until the other day. The server has been sat on my desk blinking away for months now and I pay little attention to it, but subconsciously I’ve obviously tuned into it as all of a sudden the blue LED on the Edimax WiFi dongle was no longer winking away at me.

20160305_112141

Odd! Firing up PuTTY and trying to connect bore no fruit so a quick yank of the power cable and letting it reboot seemed to fix things until it fell over again a few days later. Very Odd.

With an inability to connect to the device there was no option but to hook it to a monitor and plug a keyboard in. Again, nothing. I’ve mentioned before that I’ve had flashcards corrupt when I’ve power cycled Pi’s in an unfriendly manner so put it down to that.

There was little option but to rebuild an image and start from scratch, which is where things started to get interesting.

Now I don’t know what’s going on but when using Jessie distributions of Raspbian, when the Pi boots I’m getting all sorts of weird memory dump errors during the boot sequence which, without having a keyboard attached you’re unable to acknowledge, so the Pi is left hanging expecting a key press, which it’s never going to receive as it’s running in a headless implementation.

A lot of time was spent updating the distribution, working on the assumption of a kernel or driver issue but to no avail.

As such the only solution I’ve got for this so far is to downgrade to Wheezy.

This then causes its own problems as Jessie very nicely accepts MAC bound IP addresses from my router with very little configuration. The GUI default with Jessie means WiFi network keys can be added instantly and are saved in the various config files along with SSID’s.

Wheezy leaves you having to rummage around inside the various config files using nano to set things up which isn’t as nice but so far, touching wood, we have a functioning NTP server.

Now there’s a silver lining to this cloud! I mentioned before that the GPS module used in this project is a motion based unit rather than a static time sensor and with the necessary skills you can force it into that time mode increasing sensitivity. I never worked out how to do that but on following the Ava HAB guide again this time, the necessary code and instructions have been added!

The necessary steps for setting either stationary or portable mode are as follows –

Download the following –

wget https://dl.dropboxusercontent.com/u/63720513/Code/gpsControl.c

Compile +link –

gcc -o gpsControl gpsControl.c

As sudo issue either command for portable or stationary –

./gpsControl -p -d /dev/ttyAMA0 = Portable Mode
./gpsControl -s -d /dev/ttyAMA0 = Stationary Mode

Now the Ava HAB guide says it may take several attempts to set the mode and they’re not kidding. I repeatedly get the following response

 

M0XXF@Atlas ~/ntp-4.2.8p3 $ sudo ./gpsControl -s -d /dev/ttyAMA0
Set GPS for stationary mode
Configuring device /dev/ttyAMA0
>>>>>>>>>>>>>>>>>>>>> SENDING >>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> SENDING >>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> SENDING >>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> SENDING >>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> SENDING >>>>>>>>>>>>>>>

** FAILED to set GPS Mode**

That isn’t the end of the world as the priority was to get the NTP server back up and running.

Advertisements

How to build an Atomic Clock

OK there’s a degree of licence in the title here, but unlike the “Daily Toilet Paper” which came up with headlines such as “Freddie Star Ate My Hamster” to increase it’s circulation, there’s more truth in my tag line than meets the eye.

You’re here now so read on!

As a kid I grew up in the era of having only three TV channels which switched off at midnight after the ceremonial playing of the National Anthem. As such there was a lot more radio in the house than there is today and I was always intrigued by “The Pips” which chimed out on Radio 4 at the top of the hour. I can see some people saying “You what?”

Yep those are “The Pips” also more formally known as the Greenwich Time Signal. As a nipper they always seemed a bit of fun just before the boring news, but when my Dad told me that they were generated by a very precise clock called an Atomic Clock . . . Wow! All sorts of mad scientist James Bond evil villain lair images sprung to mind. Now as you get older those images fade (slightly) as you become more educated and grown up, but the words atomic clock still conjure up images of machines the size of the CERN super collider. Now undoubtedly at one stage of their development they were and you could ask the question what use are they and what relevance is all this to radio as a sport?

Well firstly an atomic clock uses the “oscillation frequency” of particles (the rate at which they move up and down) to provide a very accurate time. That’s a bit simplistic but have a look at Wikipedia’s entry about Atomic Clocks for a more accurate explanation.

WSPR relies on a very accurate clock to sync the transmission and reception of WSPR signals.

Hence, accurate clock = a whole world of other radio related projects!

My WSPR receiver (see earlier posts) used a PC running Meinberg NTP to decode WSPR signals. My Ultimate 3 beacon uses a GPS signal to synchronise the clock within for transmission.

My plans next year are to have a crack at some other SDR projects based around new kit on the market and as part of the background reading ahead of parting with any money I discovered that one of these devices allowed for attachment of a 10Mhz external clock reference. A few months earlier, Everyday Practical Electronics (EPE) ran a constructional project around building, wait for it, an atomic clock or more accurately a rubidium standard. I can see peoples eyes glazing here but bear with me.

All a rubidium standard does, is provide a very accurate signal pulse, in the case of the EPE project it was a 10Mhz signal which could be used as a reference source for calibrating or testing things on a work bench.

I had visions of this costing thousands and requiring a degree in atomic physics to complete. Erm no, about £125 quid and the ability to solder half a dozen wires!

$(KGrHqV,!icE8V7Bpn1PBP(BziLqIw--60_12

That my friends, is a rubidium standard. It’s about the size of a paperback book and thanks to our friends in China and the ex Soviet block there are loads of these things sloshing about on the internet. I’m sure they may well have been lurking in a bunker along with other things which would make you glow in the dark in less friendlier times, but in these days of world peace and consumerism they’re being flogged off left right and Chelsea! All you need to do is give it power and you’re off.

Now as tempting as it was to build one, I haven’t. What I have built is the next best thing, which is a precursor to other projects and is a proof of concept. It also has a host of practical applications.

What I’ve built is a GPSDO based NTP server. Again I here a “You what?” A GPS (Global Positioning System) Defined Oscillator based Network Time Protocol server.

Breaking it down further the GPS satellites orbiting the earth transmit a time signal which your satnav sat on the dashboard of your car receives, does some maths with the data inside that signal and works out where you are. That time signal is highly accurate, lets face it the US military have been using it to help bomb countries back to the stone age for decades so it must be good!

An NTP server provides computers with an accurate time reference. Have a look at Wikipedia for a proper introduction and explanation.

NTP helps keep things on track. Lets face it, without it your PC’s clock will drift significantly. If you have a rummage around in the clock settings within Windows you’ll see that Mr Gates wants your computer to use the good old Microsoft time servers by default.

ntp-setup-H1

Now that is all well and good and for the majority of people will do the job nicely, but like everything there are limitations.

NTP services are hierarchical in nature and the various levels are referred to as Stratum

350px-Network_Time_Protocol_servers_and_clients.svg

Stratum 0 -These are high-precision timekeeping devices such as atomic (caesium or rubidium) clocks, GPS clocks or other radio clocks. They generate a very accurate pulse per second signal that triggers an interrupt and timestamp on a connected computer. Stratum 0 devices are also known as reference clocks.

Stratum 1 – These are computers whose system clocks are synchronized to within a few microseconds of their attached stratum 0 devices. Stratum 1 servers may peer with other stratum 1 servers for sanity checking and backup. They are also referred to as primary time servers

Stratum 2 – These are computers that are synchronized over a network to stratum 1 servers. Often a stratum 2 computer will query several stratum 1 servers. Stratum 2 computers may also peer with other stratum 2 computers to provide more stable and robust time for all devices in the peer group.

Stratum 3 – These are computers that are synchronized to stratum 2 servers. They employ exactly the same algorithms for peering and data sampling as stratum 2, and can themselves act as servers for stratum 4 computers, and so on.

Network speed, latency, network outage and so forth will reduce how “Average Joe” at the bottom of the pile receives their NTP source. By having a Stratum 1 computer within your network you are self reliant!

So, how many thousands of pounds does this cost I here you ask? Less than £50!

The ingredients for this little project are a Raspberry Pi (B+ in my case), a Raspberry Pi B+ GPS Expansion Board and an antenna.

20151216_094931

Plumb it all together and you get

20151216_100801

Apart from it looking pretty cool it does exactly what you want very well. The GPS Expansion Board is based on a Ublox MAX-M8Q positioning module and is pretty damn accurate! Admittedly it’s a motion based unit rather than a timing module but by issuing it with a serial command, it can be placed in “Stationary” dynamic mode which is the default mode for the much more expensive timing modules. I haven’t sussed that yet but all in good time!

The AVA High Altitude Balloon Project site has a very good How To which is what this project is based upon.

Also worth a read is the satsignal.eu Raspberry Pi Quickstart and the Raspberry Pi NTP guide.

Now unfortunately there’s a cautionary tale linked to this “How To”. All the bits turned up by Monday and on Tuesday I had a spare hour before going to work and decided to get it all working. The AVA walk through worked perfectly and I had the server generating a more and more stable time signal as it settled down after finalising things. Originally I had used a network cable to speed things up rather than using WiFi. The final step was to configure the on board WiFi to give it a fixed IP address and off you go.

Having played with Raspberry Pi’s before, the WiFi networking can be a real pain and not very stable but I bit on the bullet, started the GUI and began entering SSID’s and network passwords. At this point the GUI crashed quite spectacularly, locking up the machine. Having corrupted the SD card on a previous occasion by yanking the power cable I was reluctant to undo the last hours work, so while still being able to PuTTY into the Pi I rebooted it, only to find the damn thing had corrupted the SD card. Awesome!

Not wishing to be too defeatist, knowing it worked in principle I left it alone until Friday when I had a spare hour again to start from scratch. Until that hour turned into half a day! For some strange reason the device kept corrupting, refusing to boot and powering off the on board NIC so you couldn’t get into it via SSH or by being plugged into a monitor with a keyboard!

Now this caused a considerable amount of head scratching and effing and jeffing! Setting up a Raspberry Pi is simple but not the quickest of processes. At one stage I had two machines burning Wheezy and Jessie images to SD cards in a little production line ready for when I trashed the next one!

Both the AVA guide and Satsignal.eu guide have the following primary steps which get the Raspberry Pi ready for installation of the packages which actually do the clever stuff –

sudo raspi-config
1. Expand Filesystem
2. Advanced Options -> Disable Serial Shell (optional)
Reboot

sudo apt-get update            – tells the Pi to update its list of available packages
sudo apt-get dist-upgrade – updates the Pi’s distribution to the latest version
sudo rpi-update                    – updates the Pi’s firmware to the latest version
sudo reboot                            – reboots the Pi ready for the next stage and allows the updates to finalise and so forth

sudo apt-get install pps-tools – first step of installing all the necessary bits and pieces to get the GPS side of things working

Now by process of elimination (and this wasn’t quick!) I found the Pi was dying after pps-tools was installed, which was far from helpful.

The only conclusion I could make is that between Tuesday lunchtime and Friday morning a package in the distribution upgrade or the firmware update had been altered/upgraded and released to the world, which was killing the Pi once pps-tools was installed.

Proof of concept comes in that if you omit the steps

sudo apt-get update
sudo apt-get dist-upgrade
sudo rpi-update
sudo reboot

everything works perfectly!

Now how do you prove it’s working? Good question. Once the NTP server is running, issuing an ntpq -p command gives you data

Screenshot 2015-12-20 16.42.04

  • The display is a list of remote servers with various status reports arranged in columns.
  • One remote server should have an asterisk (*) in the first column.  This marks the server which NTP has selected as the current preferred source.
  • Servers which have a plus sign (+) are good enough for NTP to sync to, others are not.
  • The reach column should not be 0, and will expand from 1 during the normal working of NTP until it reaches 377.  It is an octal display of a bit-mask showing when the server was reached.  Normally you expect to see 377 in this column against each server.  A column of all zeros means that NTP can’t contact any servers.
  • The offset shows how far your PC is off from a nominal UTC, and the value is in milliseconds.
  • The poll value should gradually increase from 64 seconds to 1024 seconds as NTP needs to contact the server less and less frequently as the clock offset and frequency are gradually corrected.  Changing the poll is automatic in NTP.
  • The delay shows the time for a packet from your PC to reach the remote server and vice versa.  Values above 150ms may indicate a satellite circuit and it’s best to avoid such servers if possible.  You will get best performance from servers which are close to you on the network.
  • The jitter column shows how stable the connection between you and the remote server is.
  • The st column shows the stratum of the server, with stratum 1 servers having a local reference such as an atomic clock or, for many servers, a radio-clock or GPS receiver reference.  Most servers you will see are at stratum 2, so they are locked to a stratum-1 server.  A lightly loaded stratum-2 server is probably a better reference than a heavily loaded stratum-1 server such as those with widely-publicised addresses.

Now that the server is working and providing good stable timings it’d be a shame not to share it! This can be done in a multitude of ways. You can add the IP address of your NTP server to you hosts file within Windows as a TIMESERVER, so when you go to your time and date settings it’s there as an option to select.

Alternatively, if you use Meinberg NTP, by adding the NTP server to the ntp.conf file your local machine then has a Stratum 1 time server available to it.

Screenshot 2015-12-20 16.46.56

I’m slowly editing all the clock settings on my network machines to allow them to utilise the local NTP server as well as trying to get my head around trying to monitor the server’s stats using MRTG. MRTG is easier said than done as it’s based on Pearl and I haven’t done any scripting in years, but that’s another rainy day project.

As I said at the beginning this is a proof of concept for another project and it works a treat!