solar

Aurora Inverter Monitoring Done Quick

Last time I set up inverter monitoring for my Aurora solar inverter, it had a lot of moving parts. I had a RaspberryPi ModelB hosting utilities writing to CSV files, which were periodically scooped up by a monitoring service. The monitoring service had a bit of setup and a few issues, but in the end it would get the data up to PVOutput.org.

This time around I decided to cut out all the bullshit and write to the PVOutput API with as little fuss as possible.

Note that to set all this up you will require a free account on PVOutput, and you will need to set up an API Key. You will also need to add your solar details which will give you a system Id.

Curtronics Aurora

Grab this great little utility - Curtronics Aurora. Download, unzip, and install:

wget http://www.curtronics.com/Solar/ftp/aurora-1.9.3.tar.gz
tar -xf aurora-1.9.3.tar.gz
cd aurora-1.9.3
make
sudo make install

At this point you should try a query and see if you can contact the inverter (keep in mind if it is dark outside the inverter is probably powered down).

Try the following command:

/usr/local/bin/aurora -a 2 /dev/ttyUSB0 -Y15 -e

If you are not having any luck, check if /dev/ttyUSB0 is correct (this is your usb-to-serial device). You should be able confirm it by reading through your start up messages with dmesg.

solar-logger.sh

Save this script in your home folder as solar-logger.sh.

If you are using the default pi user, the path will be /home/pi/solar-logger.sh

Make sure you set execute permission: chmod 755 solar-logger.sh

#!/bin/bash

# configuration
apikey="your api key here"
system="your system Id here"

rd=$(date +%Y%m%d)
rt1=$(date +%H)
rt2=$(date +%M)
rt="${rt1}%3A${rt2}"
solarout=`/usr/local/bin/aurora -a 2 /dev/ttyUSB0 -Y15 -e -c | awk '{printf $1}'`
solarwatts=`echo ${solarout} | sed -e "s/\.//"`
url="https://pvoutput.org/service/r2/addstatus.jsp?d=${rd}&t=${rt}&v1=${solarwatts}"

# only make the request if we have a value from the inverter
if [[ ! -z $solarwatts  ]]; then
    result=`curl -H "X-Rate-Limit: 1" -H "X-Pvoutput-Apikey: ${apikey}" -H "X-Pvoutput-SystemId: ${system}" ${url}`
    if [[ $result == *"OK"* ]]; then
        echo "${rd} $(date +%H:%M) : API call successful, power output: ${solarout}"
    else
        echo "${rd} $(date +%H:%M) : API call failed, ${result}"
    fi
else
    echo "${rd} $(date +%H:%M) : Inverter communication failure"
fi

Cron Job

Edit your crontab (using crontab -e) and add the following record:

*/5 5-18 * * * /home/pi/solar-logger.sh >> /home/pi/solar-log.txt

This cron task will run the solar logger every 5 minutes between the hours of 5am and 6pm. You may want to adjust those times depending on your latitude.

This will give you a solar-log.txt file with entries like this:

20191201 18:25 : API call successful, power output: 20.373
20191201 18:30 : API call successful, power output: 20.373

Once you start seeing successful API calls you should be able to see your shiny new chart on PVOutput.

pvoutput live output chart

Troubleshooting

If you are having issues, check the permissions on the aurora binary (755 should work).

/eof

OCR Solar Meter Reader - Part 1

I had some solar panels installed last year. I wanted to read my inverter statistics, but I also wanted to log exactly how much power I import/export. Challenge - I can’t physically plug anything into the meter (power company regulations). What to do?

How about a web cam, raspberry pi, and OCR?

The Goal

The goal is to get all my data to log auto-magically to the PVOutput portal (pictured). Here is my PVOutput page.

Physical Layout

The Problem

I have an array of PV panels on my roof, which are connected to an inverter. The inverter feeds generated power back into the power grid via my power meter (the power company pays me for this).

I decided I would use a Raspberry Pi to read data directly from the solar inverter. That gives me stats such as the amount of power produced by the array. It doesn’t give me how much I actually export to the power grid - as that figure comes from the power meter.

The formula for exported power is:

Exported Power = Generated Power - Power Used by my house

So, how do I interface with my EM1000 power meter? The meter has a couple of ports:

  1. RS232 Serial Link
  2. Optical Port

Should be easy right?

Not quite.

The power company seal the RS232 port. Breaking those seals is a criminal offence. I did try to get permission to do it, but my provider (Energex) kept saying no. That leaves the Optical Port. Optical Readers are expensive, but a bigger problem is that Energex doesn’t use a standard protocol. They came up with their own (figure that one out, existing protocols just aren’t good enough read a number from a meter. I wonder how many hallucinogenic mushrooms were consumed to reach that decision).

So, what to do? The only other place data was visible was the LCD. Surely we can’t just take photos and OCR (Optical Character Recognition) the result? It can’t be that simple? Well…. kind of.

Physical Diagram

Let’s skip ahead to the physical diagram, and then look at how I got there. This is the physical set up of the system:

Physical Layout

Physical Setup

Raspberry PiA power box is fairly small, so I expected a challenge fitting everything in.

Getting the Pi in there was fairly easy. A piece of velcro tape keeps it firmly in place.

The Pi’s USB port does not provide enough juice to power anything more than a keyboard, so I have added a powered USB hub. That lets us power a light and a webcam.

For the webcam, I am using a Logitech C270. Due to the short distance between the cam and the display, I had trouble getting the camera to focus. For my first attempt, I mounted the camera above the power meter and used a mirror mounted on the door to get enough focus distance. This did work, but wasn’t great, so I took apart the camera and manually refocused it. Here is a guide on how to refocus the camera: YouTube: Refocus Logitech Webcam. It is easy, but you can kiss your warranty goodbye.

With the focus fixed, I was able to mount the camera properly. I used some cheap metal strapping from Bunnings, which I bent into an “L” shape. This was screwed into the top of the meter box. To secure the camera, I removed the camera clip and then screwed the camera directly into my home made bracket.

Lighting

Lighting was a particular challenge. For the display to be readable via OCR, there needs to be a consistent brightness across the entire display. There also needs to be enough contrast to clearly differentiate the characters against the background.

Fail LightI tried a flexible stick on strip light - but there were too many light sources, and they were too bright. I then tried an el-cheapo goose-neck USB light in a “fake lightbulb” housing - but that was too inconsistant. One side was dark, the other light!

Lighting up the boxI settled on a USB goose-neck reading light (found by accident in Myer). I mounted it higher than the LED display to minimise reflections. I was nearly there, but not quite, I was still getting too much glare. My first eureka moment was spraying the inside of the power box lid with matt black paint. That cut down the glare, but I still had contrast issues. My second eureka moment was moving the camera down, so it looks up at the display on a 15 degree angle. Suddenly I had a razor sharp well lit image! nice!

Network

I tried WiFi first, but found that putting a wifi adaptor in a big metal box makes reception near impossible (not too surprising, I guess). I didn’t want to run Cat6 cable to my power box (too much 240v power in the ceiling/wall cavities around there).

As luck would have it, I had some spare Netcomm Ethernet-Over-Power EoP adaptors sitting around. I plugged one in and away we went. If you haven’t seen these before, they are really neat. They use your existing electrical wiring and give speeds anywhere up to 500Mbit (half duplex!). Like everything, they have a catch. If your electrical wiring is bad, or you try to run them across circuits - performance will be abysmal. Certain appliances can also interfere with them. Have you ever tried troubleshooting a network where turning on a kettle (jug) kills connectivity?

Part 2

The next part details the software side of the project.

/eof

OCR Solar Meter Reader - Part 2

Continuation of Raspberry Pi Solar Data Logger article. In this part I detail the software side of the OCR solar data logger. Perl, APIs, C. Hooray!

The Software Part

With the hardware part sorted, now comes the easy part - software.

Getting inverter data is easy enough, but reading the meter is more of a challenge.

There are 3 basic parts to this:

  1. Take photos
  2. OCR the photos and store the values
  3. Send it to the PVOutput.com API

We also need something to tie it all together. I’d usually use my beloved DotNet stack, but this is Linux so that means Mono… and getting Mono working on a Raspberry Pi requires some form of animal sacrifice… or a prayer to Zeus… or something.

I considered using shell script, but then I remembered how much I hate shell scripting. I fired up my editor and hacked up some Perl. Why Perl? Perl is tried and tested, has good library support, and is available pretty much everywhere. Also, this is valid perl code which amuses me far more than it should:


    print'````'^RPQT

How can you not love a language where that is valid code? source. The only language sillier than that is BrainF*ck… god help anyone trying to do something productive with that.

Process Overview

Here is an flow chart of the process.

Physical Layout

Essentially, the pseudo-code version goes like this:

At 11:45pm take 20 photos
For each photo
    OCR photo
    If (value -ve) store export
    Else store import
If we don't have import/export explode, and notify admin
Store import/export in CSV
Calculate daily import/export by subtracting previous days values
Send daily import/export to PVI Output REST API

Taking Photos

I found a handy little command line tool for taking snaps - uvccapture. All it needs is a camera location and resolution and away it goes.

This is a photo taken with uvccapture:

Web cam photo

OCR time!

Figuring out how to OCR the display proved a bit tougher than I’d hoped. I found this great little app -Seven Segment Optical Character Recognition, SSOCR for short.

SSOCR was developed with the intent of reading RSA security tokens, allowing one token to be easily shared by a group of people - which completely circumvents the point of having a RSA token.

The gentleman who wrote it is a stern German who appears to still live in 2005, in a world devoid of joy, happiness, and code sharing tools such as GitHub. He’s aware of GitHub, but refuses to use it. I’m sure he’s a very nice guy, but he needs to be slapped into 2014.

Moving along.

SSOCR is written in C, and has the source code available - allowing us to modify it to work with something other than an ancient RSA token.

The changes I had to make are:

  • option to ignore decimal points
  • option to allow minus signs (for negative numbers). Only the first character is checked.
  • option to allow blank characters
  • recognise a 9 with the lower segment not set

I emailed the stern German with a list of the changes I made, but neglected to email my patch through (I did intend to!). In efficient German fashion he has implemented all of those things on his own (they are all in the latest version of SSOCR).

SSOCR is really cool. It scans the image, line by line, and attempts to identify line segments. It then uses the segment data to figure out which number was discovered.

Here is some debug output from SSOCR:

OCR Output

The different coloured lines show where SSOCR has identified line segments. Neat.

Once I have the SSOCR output, my perl script store it in a CSV file.

Hello, PVOutput API

PVOutput provide a handy REST API for pushing your data in. As much as REST is straightforward enough, why can’t the damn thing provide a WSDL like SOAP? instead, I have to type out all my requests like some sort of neandrathal. On the plus side, I don’t have to deal with 3 miles of XML gibberish, so there’s that.

This is how easy it is to submit data:


    my $ua = LWP::UserAgent->new;
    $ua->default_header(
                    "X-Pvoutput-Apikey" => $pvoutput_apikey,
                    "X-Pvoutput-SystemId" => $pvoutput_sysid,
                    "Content-Type" => "application/x-www-form-urlencoded");

    my $pvoutput_url = "http://pvoutput.org/service/r2/addoutput.jsp";
    my $request = POST $pvoutput_url, [ d => $date, e => $export*1000), ip => ($import*1000) ];

    my $res = $ua->request($request);

    if (! $res->is_success) {
        die "Couldn't submit data to pvoutput.org:" . $res->status_line . "\n";
    }

There is another reason to love perl. If things go wrong, we don’t exit, or error like lesser languages. No, we commit seppiku. “I can’t fulfil this task you ask of me, so I’m going to die”.

Perl is great.

Getting Inverter Data

I use Curtronics Aurora program to retrieve statistics from the inverter, once per minute. These statistics are logged and stored in daily CSV files.

Data is sent to PVOutput every two minutes, using the pvoutput-integration-service.

Once running, this service is set and forget. Just remember to add a start up script for it, so it comes back up if the Pi goes down for any reason.

So, between the pvoutput integration service - and my OCR magic above, both the inverter data and meter data are being sent to the pvoutput portal. Hooray!

When it all goes wrong

Did you know the Raspberry Pi has a hardware Watchdog Timer? Overview:

A watchdog timer is a special kind of timer commonly found on embedded systems that is used to detect when the running software is hung up on some task. The watchdog timer is basically a countdown timer that counts from some initial value down to zero. When zero is reached, the watchdog timer understands that the system is hung up and resets it.

The quote above is from Ricardo’s Workbench. He explains how to run it on your Pi. So, now my Pi reboots itself if it hangs.

Very nice.

Conclusion

It has taken a long time, and lots of experimenting to get the solar meter reader working reliably. I’ve had a lot of fun doing it, and learned a few things.

I went up the garden path a few times. For example: when I was having trouble getting the OCR to work reliably, I wrote a Node.js based site for selecting photos from the meter and updating pvoutput. That’s a future blog post.

I’m really blown away by how far things have come in the last few years. x86 PCs the size of a mouse? Incredible. I can’t wait to see what the next few years bring.

/eof