Raspberry Pi -Teaching

I know its been a while since I have posted something on the blog and I do apologise for that. But I have been teaching some instruments technicians and preparations for class is time consuming. I have also spread the raspberry pi fever to my students. The raspberry pi is now the official teaching device for the students (along with python) and  I am pleased with their progress and enthusiasm. I have taught them lots of new exciting but its more satisfying for me when I learn new things and I have learnt some new techniques when it comes to parsing strings.

Despite my busy schedule, on my down (when not preparing for class) I still have been continuing my work with the raspberry pi. I have looked into some php scripting, databases (hosting and pushing data to remote databases)  and last and by no means least, the Raspberry Pi camera. Look forward to those blog posts but I have a few others I have in store as well...

Needless to say I  very excited about the Raspberry Pi and its vast application uses. However, up to now I have not worked with the GPIO pins in a project. Yes I have turned on an LED but to make it into a practical project I have not gotten around to that as yet. I am thinking that 3.3V lcd screen sitting in my office is calling my name.

Until next time...

Marvin...

The Chronicles of RPi - The Raspberry Pi

So far I have tried really hard not to mention the Raspberry Pi (RPi) in my blogs. I wanted to give a little background about the work I was doing and until recently, all these projects have been running on a PC.

I won't go into much detail about the hardware and software of the RPi,  but it's basically an ARM processor running a linux distro (Raspian in my case). Currently,  there are two versions of the RPi, versions A and B, but I have been using version B in my projects, which has all the whistles and bells.

For further information about the raspberry pi, visit, www.raspberrypi.org

Once you have an appreciation for the RPi, not what it is capable of but more so its size and cost, so you can see why I was so excited about this project. Essentially,  l replaced a PC system, worth a few hundred dollars, with a $50 dollar linux computer. This excitement has spread to others at my work place, as they are asking if they can include units, so they too can experiment,  in the next RPi order.

The work mentioned in the blog thus far (see The Chronicles of RPi - Web Hosting for an update)  is currently running on a RPi and it has been very reliable. I have had some issues with the network at work where it goes down, making the RPi innaccesible over the network but once connection is reestablished,  all the data is present and accounted for. I primarily communicate with the RPi over an SSH connection and sometimes a reset is needed of a network switch and seldomly the RPi itself.

I am constantly making the scripts more robust and efficient as I plan to deploy the unit at another off site location. Another "problem" I have to deal with is its packaging. Its cost alone can exceed the cost of the unit so I am trying to find a cost effective to do this. Many cases are available online but they focus primarily on securing the RPi and not its peripherals. This could take some time so a big box is probably going to have to work for now.

While making adjustsments to the ERC project python script, I deployed a RPi in our second cluster room at work for temperature monitoring and alerting purposes. This will be the topic of my next installment.

Until next time.

Marvin...

The Chronicles of RPi - Web Hosting

Before I begin to focus on this blog, let me recap what has been done. So far we have a:

• UNIX based system communicating with a commercial AWS and retrieving the attached sensor readings

• Saves the data  internally (this was not explicitly stated)

• Displays the data in an HTML table (you may want to number or bullet these)

• And finally plots some of the data on graphs and saves the graphs as an image (in my case .png format)

All this is being done by a python script being executed by a cron job every minute.

Ok good.. As the title suggests, "Web Hosting", I decided to take the data dissemination to the next level because all the above mentioned options were limited to the single station for viewing. At the Institute (my work place) this type of data is understandably in high demand, so web hosting the data on our internal network seemed like the logical thing to do. This would also prevent people from calling me asking for data.

I will not get into the details of which folder contains which files etc here, although I may probably do that in a later blog when I am breaking down the different aspects of this series of blogs. I will say that to host a web page you at least need some form of software to do this. The most common one and the first one I used was Apache. Once installed, it creates the folder(s) needed where by default web hosting content should be stored. These paths I guess can be changed but why would you want to do that.

I am by no means an expert when it comes to HTML scripting but there are lots of sites that can assist you with the very basics. I did some basic HTML coding before but I cant remember if that was part of a university course or on my own for personal fulfilment.

Ok. So I have an HTML page which displays a table of data and I have images in .png format and a web server installed on a UNIX based system. Could this recipe come together any better?

All that is needed now  is some creativity and the structuring of what I want shown. I don't think I have a graphical design or artistic bone in my body or it was shattered into tiny pieces after getting 3 out of 10 on my art home work I spent nearly a week on and some guy who forgot to do his drew something in less than 5 minutes got 7 out of 10. Ah well... moving on, I opted to go with the standard white background for the table and the same for the images. I guess the content can speak for itself.

One thing I will say about web serving software, is that they usually default to index.html scripts. That's what they consider home base and any folders created in the original web hosting folder would just be a "/" away. Having said that, I created two folders with in the web hosting folder "student" and "aws" and within them placed the HTML scripts. The HTML table script is named "observer.html" within the "aws" folder. To access it from within our internal network you just type, "192.169.0.87/aws/observer.html" in any web browser  and that will pull up the HTML page which contains the table. (providing that you are on the same sub net mask as the UNIX system)

To access the images, I created a HTML script named "index.html" and saved that in the "student" folder along with the images I would like to display. The content of the  "index.html" script I will discuss in detail in a later blog. To access the images from the internal network you just type, "192.168.0.87/student". As you can see I did not have to write the HTML script name in the address in this case.

I was amazed at how easy it was to host a web page and since then I have made it common place in my work to have all of my projects do some sort of web page hosting for information, analysis or diagnosis purposes. I now have a page that links to several web pages that my projects hosts and its the first thing I check on mornings when I get to work. Just to make sure all is working as it should. 

Picture of Maintenance Web Page which links to my other project's web pages

Once, when I was away on work travel,  even implemented a system where some generated images were sent to out FTP serer. I then had an FTP client programmed installed on my android tablet and that save the images that were stored on the FTP server. Along with an HTML page viewer software and a basic HTML script. I was able to monitor my work while periodically in the meeting.

I guess we have the birth of a future blog but it won't be the next one.

Until next time.

Marvin...

The Chronicles of RPi - GnuPlot

After the HTML table display, I decided to take it one step further and plot the data in graphical format. In keeping with my open source ideology, I decided to use GnuPlot, which is a command-line graphing tool. Another reason for using GnuPlot is that it had a Python library which would fit in nicely with the existing script.

Presently, only temperature, dew point, relative humidity, pressure and AWS battery voltage (for diagnostic/performance reasons) graphs are being generated from the incoming data. There are three possible types of graphs:

1. A current graph - this shows the data set with it time axis close to or at present time at its extremity.

2. A 24 hour graph - this shows the data set on a full 24 hour time scale. Its easier to spot trends on this graph than on the "current graph" . Sometime these graphs can look different although the data being presented in both is identical.

3. A weekly graph - this is where the data is displayed cumulatively for a week from its present time. Therefore, any data older than a week will be discarded. (NB. This graph is data dependent and not time dependent, if there are any significant breaks in the data set, it is possible that the time scale can read more than a calendar week.)

Following closely behind  these variables will be rainfall. That format has been identified and will be implemented in the near future.

An example of a "Current Graph" of Temperature and Dew Point

GnuPlot is a very versatile graphing tool with many features. Two of these features that made my work a lot easier were its "NaN" and is ability to output the graphs as a .jpg or .png formatted image.

The "NaN", which stands for 'Not a Number" feature allows me to some rudimentary quality control checks in my scripts and flag and "weird" data and then prevents that data from being plotted in the graphs. Once flagged, GnuPlot will simply omit that value from the graph and leave it blank. These QC checks are hardly used but if implemented, the are usually one out of every 100 readings and on the entire scale of the graph, it is usually not seen.

The formatted image feature usefulness will be discussed in the next blog as I look into the world of web hosting the data.

Until next time...

Marvin...

The Chronicles of RPi - Its History

Under the ERC project a data collection platform was installed at CIMH. This displays data of all formats on a Google Maps based interface.

The main involvement for me was to write scripts, in Python,  that were used to parse out AWS (Automatic Weather Station) data from Windows based computers and send them to an ftp server using the sftp protocol. Since the AWS collection systems were Windows based, I had to install CYGWIN on them to implement a unix environment to take advantage of cron jobs. CYGWIN is good but I have my reservations over prolonged use of it, especially on more modern Windows systems. 

As anticipated, CYGWIN did not perform satisfactorily. Surprise surprise. CYGWIN + Windows 7 = Headache.

This then prompted me to find an entirely UNIX based solution after numerous complaints and countless system restarts. After all, the data collected from these stations and their eventual display on the platform rested squarely on my shoulders. (No pressure)

After weeks of sifting through user manuals and adjusting scripts,  I was able to communicate directly to the AWS data logger and retrieve the data. At this stage I was then able to implement some of the original snippets of code to complete the job. The original script not only parsed out the data for sftp upload but also made the data visible on an HTML page in a tabular format. This was a bit of forward thinking on my part as I envisioned issues arrising from the CYGWIN / Windows 7 combination.

HTML Table Displaying Attached Sensor Data

This is where it all began.

Until next time...

Marvin...

The Chronicles of RPi - The Beginning

This blog will be a soliloquy of my works in discovering the wonderful world of the Raspberry Pi (RPi).

Stay tuned to read more about my adventures.

Until next time.

Marvin...