Tweeting the weather with a WS2350, Open2300 and Raspberry Pi

13 06 2012

Well the last day or two have brought another USB/Serial adapter and some success, the weather station (Ws2350) is now communicating with the Raspberry Pi via USB. So what was the problem and what was the solution?

The weather station is well know for being a picky little device (read non standard) about its serial comunications, you’ll find many horror stories from people who have had to link it via a USB adapter. It’s worth noting that even though I now have communications there may be further problems to come, so the side project of adding a serial port to the GPIO of the Raspberry Pi is only on hold.

Anyway the problem was having a USB/Serial adapter that both the Raspberry Pi and the weather station were happy with. Other people with similar issues have thought it may be power related as the Raspberry Pi doesn’t have a lot of power to pass to the USB, but these adapters draw less than 25mA and I tried via a powered USB hub too, it wasn’t a power issue.

Original (supplied with the weather station) FTDI adapter (FT232BM), worked with the weather station and using Ubuntu on a PC, hung the Raspberry Pi when accessed but was recognised by the Raspberry Pi when plugged in.

PL2323 Adapater, bought to try a different chipset. Raspberry Pi was quite happy with it, even hooked up a PSION Series 5 as a terminal (see this) but the weather station via Raspberry Pi or via Ubuntu on a PC did not want to play. This failure was almost certainly down to the WS2350 being picky about its adapters or possibly a non standard pin out on the adapter.

FTDI Adapter (FT232RL) I had a post about the problem at RasberryPi.org and someone confirmed that they had an FTDI device working with Raspberry Pi, a device using a newer chipset. I did some research at the chip manufacturer and decided to track down an adapter using the newer FT232RL chip, thinking that as the weather station was happy with the old FTDI adapter maybe it would be with the new one too. I found one at tronisoft.com and ordered it on Sunday evening, it arrived Tuesday am. It worked.

Now that communications were in full swing I started to hack a little bit of python code to read the Open2300 log file and tweet it, my current Cumulus software tweets the weather every hour or so and its pretty easy to tweet from code.

 

The tweet2300.py code

#
#tweet2300 Steve Wardell 2012
#
from subprocess import call
import twitter
#auth twitter
auth = twitter.OAuth("your", "twitter", "oauth", "keys")
t = twitter.Twitter(auth=auth)
t.account.verify_credentials()
#run log2300
call(["rm", "log2300.txt"])
call(["log2300", "log2300.txt"])
#open and read the tweet.txt file produced by log2300
file = open("log2300.txt","r")
log2300 = file.read()
file.close()
#chop up the string to get only the basics we are interested in
words = log2300.split()
tweet = "Forecast " + words[17] + " : Wind " + words[8] + "mph " + words[10] + " : Baro " + words[15] + "hpa " + words[16] + " : Temp " + words[4] + "c" + " : Rain " + words[13] + "mm " + " : Humidity " + words[7] + "%"
#tweet the content of tweet.txt
t.statuses.update(status=tweet)
#the end

Its not very elegant but as a quick hack it did the job.

Open2300 is a suite of programs to read data from the Ws2350 by Kenneth Lavrsen (his web site). For this quick hack we first get authorised with twitter then the Log2300 program writes (appends in fact) to a file a short data stream from the Ws2350. As I just want a file with a single data set the code above first deletes any existing log then writes a new one. The log is then read into a python variable and split into an array, the array is then mashed up into a suitable order with some additional text to form a tweet.

All very basic stuff and quite a bodge but we get a tweet

This simple script could be added to a CRON job to run every hour and keep on tweeting. Now that I’m happy with basic communications and have a little bit of code and a tweet to show for my efforts I can focus on developing some python routines to make use of Open2300’s fetch2300 program to write feeds for my web site weather.60redwood.com and the live gauges.

More posts as progress is made





The Weather with Raspberry Pi – unsettled but brightening up later

9 06 2012

Well, my Raspberry Pi weather station project has been stumbling along. So, first, a little update on where I am at with it.

pywws doesn’t work with my weather station, ws2350, not unexpected and a quick chat with its author confirmed it. There are a few other weather enthusiasts out thee who are trying to get it working on the RPi so check on the forums. I will be keeping an eye on progress as getting basic comms working is a common problem, as you will see….

Open2300 has been most successful at this point, its a simple set of programs that will obtain and decode the weather station data and help you use it, there is even a program that will update Weather Underground for you. Compilation on RPi did kick out some warnigns, but I had similar warnings when I compiled it on Ubuntu. The bad news, I am still not able to confirm this works on RPI, because…

Serial/USB converters – as you would expect, take two well defined standards, the latter of which was ushered in as the replacement for the former and connect them together, yeah, it was never going to just work was it! Lets be clear this is NOT a RPi issue, it’s bigger than that.

So my weather station, an entry level consumer product, about a hundred quid, has a serial link. It also comes with a serial to USB adapter as very few modern computers come with an old RS232 serial port. The USB adapter is well known as a source of endless data trouble with even the most reliable connections producing corrupt data from time to time.

Plugging this adapter, which identifies itself as FTDI, seems fine, its recognised but trying to run Open2300 with it hangs the RPi. hard to know which bit is at fault or if all of it is contributing. Running the same adapter and Open2300 under Ubuntu (on a PC not RPi) works fine though, the weather station data is retrieved and decoded.. Using a powered USB hub makes no difference in this case, the adapter is drawing around 25ma at most so power shouldn’t be the issue anyway.

To see if it was the weather station on the end of the wire causing the problem I dusted off a Psion Series 5 and serial cable, plugged it in to the RPi via said adapter and tried the terminal program, nothing, dead, hung RPi.

A second adapter arrived this morning, this time using the Prolifc PL2303 chipset (or rip off of), first off, as it was still sat on the desk, I plugged in the Psion Series 5 and, LIFE! The terminal app on the Psion communicated with the RPi.

Encouraged by some communications success I plugged in the weather station and tried Open2300, nothing for a few minutes then a time out of sorts. Moving the test to an Ubuntu PC produced the same disappointing result, it seems that although RPi is happy with the PL2303 the weather station is not.

So two different types (chipset) of Serial/USB converters and different failures. At this point I started to look for any other adapter types but on such cheap items expecting a datasheet is a little too much.

Clearly the combination of things is a key and finding an adapter that both the RPi and the WS2350 are happy with is critical, it could also be like looking for a very specific bit of hay in a haystack (I always thought that looking for a needle in a haystack sounded quite do abel to me, big magnet, little needle, problem solved)

It was time to start exploring alternative options and RPi is not without some, namely the GPIO header.

Next step, obtain or build an RS232 interface for the GPIO header, the good news is others are way ahead of me and it looks like it should be fairly straight forward, though I cant find anything pre-made for the RPi, yet.

After some hours trying to find out more about the issue via the RPi forums, lots of folks with similar issues, and more generally looking for Debian, Arm, FTDI and so on it would seem that the FTDI drivers are not ARM compatible, so if this is correct the FTDI based adapters are not going to work. I think them being identified by RPi when plugged in and checked means nothing at this point as that’s not using the driver it’s just listing the USB data, but I don’t know.





Raspberry Pi and the weather

29 05 2012

Ever since I heard about the Raspberry Pi project I knew that it would potentially be the ideal always on low power ‘pc’ to hook up to a weather station and fire data off to a web site. Realising that project has taken its first step with delivery of my first Raspberry Pi this week.

RaspberryPi

RaspberryPi

In outline the project is simple enough, use the Raspberry Pi device with a combination of pywws and Open2300 to read the data from my WS2350 weather station and ftp a data stream to my web site. Currently this is done using the excellent Cumulus software from Sandaysoft but this runs on a Windows based PC and finding a mini pc suitable isn’t cheap (which I regard as one of the main tenets of a hobby).

Currently my web site is built using various Cumulus bits but the principle is that Cumulus feeds a simple, structured text file every 15 seconds or so, via ftp, and the web site reads it and uses some basic JavaScript to dynamically update the page. A principle requirement of this project is that it either writes a compatible text file (in fact various text files for gauges and graphs etc) or I code a web site based converter that can take the Raspberry Pi feed and translate it.

Stage 1 will be to get the Raspberry Pi up and running with a stable OS, this should be fairly straightforward

Stage 2 will be interfacing the Ws2350 to the Raspberry Pi via USB (possibly via serial later)

Stage 3 will be seeing if pywws will run and read the ws2350, if not then some lower level investigation using Open2300 is going to be needed

Stage 4 will be to extract from Stage 3 a suitable file on a regular (cron) basis and ftp it to a web server (the web server will initially be one in my office, it may even be on the Raspberry Pi or another Raspberry Pi)

Each stage will be liberally inter spaced with as yet undefined quantities of alcohol which will be at various times proportional to success or inversely proportional to failure.

If any fellow weather geeks want to help out or just see how it goes I will try to blog progress here and no doubt on twitter via @stevewardell. If anyone has any ideas or suggestions or anything that may help this little venture I would be interested in hearing from you.

Let the hacking commence…

Update 30/5/12

Stage 1 held up a little awaiting some essential parts that I forgot to order initially then forgot again when I eventually got a shipping notice. Anyway, parts en route. I started to look in more detail at pywws and a few other similarly minded weather geeks have embarked on similar projects, so far it looks like the usb libs in the debian distro are dropping packets on the Raspberry Pi so there may be some issues there, also I still don’t know if pywws will even read my weather station at all.

In order to get things moving and determine if pywws was an option I set up on one of my Ubuntu servers in the office, after some fiddling to get the weather station recognised via usb/serial converter (Raspberry Pi has USB) and a few hours playing with pywws I’ve concluded it either doesn’t work with a WS2350 or I cannot get the right combination’s of usb libs to play with python. So pywws question passed to its author, in the meantime Open2300 checked out and compiled, a little configuration and success, a simple cron job has the weather station updating my Weather Underground site, all mostly thanks to the Open2300 code I have to say.

If I can decode the full memory map from the station with Open2300, which I think I can, then this looks like it should work out quite well, assuming it can be compiled on the RPi ofcourse





Weather on the move

8 05 2010

There are a few weather applications and tools I’ve come across that I have found useful to have on my phone, a HTC HD2. The HD2 runs Windows Mobile, regarded by many as one of weakest mobile operating systems available, thats not something I would argue with but its not completely useless (lets be honest Microsoft never do a complete job of anything!).

I’ve little doubt that the iPhone and Android are covered with many more and likely better applications but this is for those of us with Windows Mobile.

I’ve mostly been looking at applications rather than web sites which have a mobile version, but there are a few of those here too.

So what is there for those of us with more than a passing interest in the weather or an obsession with needing as much technology as possible to tell us its raining when simply accepting we are getting wet is not enough.

1. HTC Sense Weather

The HD2 (thank God) layers HTC Sense (Touch Flo 3D in a former life) over the awful Windows Mobile 6.5 UI, without this there would be no point really. One of the ‘built in’ applications is HTC Weather. This is quite a nice graphical tool and adds much needed eye candy to the home screen as well as offering its own application screen. You can add multiple cities as well as have it automatically report on your current location, however, the current location can be troublesome as it relies on Cell Towers for its location fix and not the inbuilt GPS, perhaps someone at HTC can offer some explanation of why?

A 5 day (today and next 4 days) forecast is the basic offering and the graphics are well made, smooth and do impress, something WinMo based phones need to do in the company of their iPhone and Android counterparts.

2. GPS Enabled Weather

This was a real find in my opinion, you can find it on their web site. The application gives you rain radar images and can take its location fix from either GPS or cell towers, GPS obviously provides a more accurate fix and avoids the ‘reception’ issues that the standard HTC Sense Weather can sometimes suffer. This is not so much weather eye candy as it is good solid weather information in the form of radar maps which you can even run an animated loop. You can set your location manually and you can touch the map to update the location too.

Two nice features are the ability to set the map zoom level (no pinch to zoom though, why not?) and a feature that allows you to call a URL with parameters passed from the application, such as your GPS coordinates. This would be ideal to call the Weather Underground mobile pages for deeper weather info on your location. Again, something I will be playing with in a spare moment.

3. iCumulus

Not so much an application as a mobile version of my weather web site created using Cumulus from SandaySoft and iCumulus from David Jamieson, written for the iPhone it seems to work fine within the Opera browser on the HD2.

I think the scripting and CSS needs some anti Safari or more Opera friendly tweaking though, something I hope to get around to soon.

As iCumulus requires a data feed supplied by your own personal weather station and web site its not something you can use unless you either have your own or access to such a site in your location

4. Touch Weather

This is very nice eye candy and solidly built, available from their web site. It allows you to select from different weather services and report on multiple locations. The basic version (free) does not have the 5 day forecast, for that you need to buy the pro version ( $5).

The default page is lacking any real detail, just the eye candy animations and the temperature. A panel appears at the bottom, when touched for, with further detail. Scrolling up and down steps you through the time period forecasts for that day, scrolling left and right takes you forward and backward through the 5 days of forecasts. A 5 day forecast summary panel is available from the detail panel too.

Its all very nice and there is a lot of customisation possible, from changing backgrounds to defining the levels of opacity used by the animations.

5. Weather4Me

The light weight Weather4me, offers decent 5 day forecasting with tabs so you can switch between Current, Today and 5 Day view. The AccuWeather tab opens an embedded browser page on the AccuWeather site for your selected location, this gives you more options with the forecast data, right down to hourly forecasts. There are also satellite images with animations.

The nice thing about being graphically light is that it will run well on other, less powerful, windows mobile phones.

The downside is that it looks less appealing next to some of the others, however, accurate data is always going to be worth more than pretty pictures.

6. Weather Underground

i.wund.com, although written for the iPhone, provides  a lot of the great data from Weather Underground in a mobile digestible format. Current conditions, forecasts and maps are all available.

The really nice thing with i.wund.com is that you not only have the choice of weather reports from all the usual sources you can also drill down to your own personal weather station, if you run one, providing you supply a data feed to Weather Underground





Redwood Lake Weather Station

2 04 2010

Over the last month of so I’ve been making a number of incremental changes to the web site for the Redwood Lake Weather Station. I still would love to build my own site from the ground up and a few prototypes have even gone from white board to web site and back again.

My biggest issue is instrumentation or gauges, Ive tried various offerings the most successful being the free gauges offered by Bindows but even these have left a lot to be desired, wide browser compatibility being top of a list of niggles.

So I decided, while I continue my search for the tools to build my own site, to apply a lot of the other tricks, tips and technology to my Cumulus based site from SandaySoft?

My weather station, a WS2350, is connected to an old server here in my office and via the excellent Cumulus software from Steve at  SandaySoft keeps the Redwood web site up to date.

Redwood Lake Weather Station Web Site

Redwood Lake Weather Station Web Site

So what are the key differences between the standard Cumulus web page and the Redwood site?

The old multi page standard site has been replaced by a single main page incorporating some of the data found on the old additional pages and driven by AJAX based dynamic updates based on a live data feed from the weather station. You can find details of the AJAX scripts on the Cumulus forum pages, implementing it was very simple.

The page has been reformatted a little with some minor tweaks to the CSS and the addition of a banner image (a picture of Redwood Lake, of course).

I have also implemented a data log to drive some graphs. The graphs are based on JPGraph package using the logging and data generation scripts by TNET Services, these are really good and very simple to get working. However they dont seem to deal with data gaps as I would like, I would prefer to see time on the graphs with no data rather than the period omitted. I’ve been looking at Open Flash Chart but have not really had the free time to define scripts to feed data in the JSON format required.

The navigation bar has been changed to allow me to include some great external content from Weather Underground and from Meteorologica, the standard Cumulus instrumentation gauges have been retained but I’m no fan of Silverlight and at best they are troublesome, which is a shame as I think Steve from SandaySoft has actually done a great job in building them.

One thing I always disliked about the standard Cumulus web site was the Moon phase graphic, so I’ve replaced it with some new images.

Overall I’m fairly pleased with the results, I think there is still a lot that could be done (time permitting of course). I would like to implement a better forecast, currently I’m using the simple (Zambretti based) forecast produced by Cumulus, the WS2350 doesn’t produce anything better itself as some weather stations do.

I think a forecast graphic and, if I can work out the scripting, a ‘current’ conditions graphic are next on the TO-DO list.





#UKSNOW gives way to #UKFLOODS

17 01 2010

With the inevitability of Simon Cowel’s next awful superstar (or RATM gaining a few hundred thousand purchasers of their records who will never play it let alone ever buy another one) we find our selves on the verge of #UKSNOW giving way to #UKFLOODS.

Here at Redwood the snow has almost completely melted, the lake has a thin covering of broken ice over most of it but its melting fast. The swans and ducks look a little happier about getting their habitat back. The lake often rises quite a bit after heavy rain and over our time here I would say we have seen it crawl up the bank by 2 to 3 feet. But the lake is not really the concern, the Loddon is.

The Loddon floods every year, certainly every year I have lived in this part of Berkshire I have read or seen it flood, with quite often disastrous results for some. Just before Christmas the following Flood Warning was issued, as far as I know its still in place. It did seem amusing to drive past the cinema and see people’s cars under water, not for the owners though I suspect.

The snow certainly caused chaos here over the last 10 days but the snow melt arriving is going to be worse. You can check flood warnings with the Environment Agency web site.

I will be keeping an eye on the local warnings, though thankfully we are high enough to not have to worry about it effecting us directly, at least for now

It seems the BBC has us down for another snow fall mid week, according to their forecast

Redwood Weather

Posted via email from Steve’s Blog





#uksnow – Snow and the Weather Station

8 01 2010

As every one not living under a rock will know the UK has been hit by quite a lot of snow in the last week. Here at Redwood we still have 8 to 10 inches laying on the ground, now frozen as we have not see a temperature above freezing for a few days.

I’ve been watching the weather station sensors of my WS2350 out on the roof outside my office windows and its stood remarkably well to its second winter in the snow. The wind speed anemometer has not frozen and has continued to provide good readings, The temperature, pressure and humidity have provided readings as accurate as they are ever going to be and well with in the margin of error when cross checked with other local weather stations. In fact only the rain sensor lets the side down a little by having no way to distinguish between snow melt and rain fall, hardly the sensors fault I know, but it does raise the question of how one avoids measuring snow melt in the rain fall stats. Its also bound to suffer from frozen water on the tipper which when it melts will add to rain fall stats too, less of a snow issue, more a freezing temperature issue.

I was also thinking about how to deal with recording snow fall, both depth and rate. Obviously there is no sensor to record this automatically (that I am aware of) so one would need to take manual readings and record these. The excellent Cumulus software (www.sandaysoft.com) I use as my data logger and web site feeder has provision for a weather diary in which snow conditions can be recored but it does not seem like the standard Cumulus site presents this diary. Of course snow fall is one thing, sunshine is another and lightning strike, both of which can be measured by the amateur using available sensors and software. Snow, it would seem, is not so easy to measure with automation.

There was an excellent series of weather programs on the BBC Four earlier this year, The Weather (www.bbc.co.uk), in four parts. One of which covered snow in some detail. I was hoping there may be a repeat showing but I cant find them.

The excellent #uksnow twitter tag mashup by Ben Marsh (uksnow.benmarsh.co.uk) has been incredibly popular again, showing a simple picture of snowfall across the UK based on a tweet comprising your postcode then a 1 to 10 scale of snowfall. But it only records snow fall not depth (in fact it only records what you tell it so its very subjective). I think there are definitely a few changes that would take it forward and make it even more popular.

The Weather Underground (www.wunderground.com) has a fantastic mashup map, based on personal weather station feeds, such as the one here at Redwood, but again, snow is obvious by omission only.

I’m sure in places where snow fall is greater that recording and publishing methods are much further advanced than they are for the typical personal weather station, but I am curious what else I can do. Its likely that these winter conditions are indicative of future winters and that snow fall will be a useful addition to the recording going on here at Redwood.

You can follow the weather here at Redwood on the web site (weather.60redwood.com) and on twitter @RedwoodWeather

There is more snow expected this weekend

Finally, here a few pictures from my office window out over the frozen garden Redwood lake

Posted via email from Steve’s Blog





The Weather

12 10 2009

I’ve always had an interest in the weather, but then, don’t we all? Except, I’ve always been interested in how it works, what it does and why it does it, as well as figuring out how to tell what it might do next. Weather forecasting is.. well ask Michael Fish!

So, a little  over a year ago I installed a weather station. This is made up of a base station which sits on my desk and via large display tells me what the weather is doing now and what it has done historically, it also attempts to provide a basic forecast. It does this by taking measurements from a weather station outside, this has sensors for

  • Temperature
  • Pressure
  • Humidity
  • Wind Speed
  • Rain Fall

The weather station is a WS2350, it is sold under a few different brands and is generally around the £110.00 mark. This price point is pretty much the entry level for a station that can be linked to a computer. In the 14 months its been working Ive not had any problems with it.

At first I used the wireless feature to link the sensors to the base, the obvious benefit being no drilling holes into the house for wires. However, there are two downsides to this. First it means the sensors have to be battery powered, not a big problem really, a decent set of batteries ran for 6 months. The second issue is more important, when connected wirelessly the sensors only take wind readings every 120 seconds or so, when connected with a wire its 8 seconds. This I found proved a bit of a limitation so quite recently my office wall acquired a hole and now base and sensors talk over the wire.

Being a tech junkie this couldn’t possibly be enough though, could it? No, of course not, we need to grab all that data and publish it online.  At first I used the software that came with the weather station I bought, this proved to be very limiting and unreliable. Searching the interweb revealed various other software systems. First I tried WUHU which is the software from Weather Underground, this allowed me to grab the data and publish it to different web sites. I still publish to www.wunderground.com and also publish a web cam image looking out over the garden.

The Weather Underground web site is very good and the features and maps are fantastic, but its not ‘my’ web site. Further trawling through the interweb revealed Cumulus from SandaySoft. This is a piece of donationware, free but if you find it useful you are asked by Steve, the author, to make a donation to fund further development. I’ve dealt with Steve via the forums he provides for help and support and both he and the Cumulus software are first class. In my opinion if you are going to set up a weather station, you use Cumulus.

So, my weather website. Well its currently more or less the default site generated by the excellent Cumulus software. Its ‘live’ most days but I tend not to run the PC that uploads the site and data all night. The site updates every minute and there is a weather feed via twitter @RedwoodWeather








Follow

Get every new post delivered to your Inbox.