I simply piped your first XML fragment in as a file to xap-currentcost and I managed to get a result, this is how I tested as I don't have an EDF unit (nor a livebox for that matter). Do make sure you have configured a CC128 unit otherwise the baud rate will not be right and you'll get nothing.
If you want to run this in debug mode -d 6 and send me some logs (as a ZIP attachment please) I can take a look at why data is not being parsed for you.
This note pertains to the RAW XML from the EDF Unit: If you capture a decent interval of data (say 5 minutes worth) and compress and ZIP attach I will have a better sample of data to test and work with. A single fragment QA does not make.
I can then fully support sensors and the like and make web page amendment to integrate this unit more natively.
Well that explain why you are seeing nothing! No data is coming into the process.
Drop down to Level 7 and you might see something, the serial input data is only displayed at these deeper levels of debugging, but you are going to get a LOT more information too.
I've push a new BETA 302.6 which now has native support for the EDF unit. You will need to select this from the WEB GUI before things will work correctly. I've also incorporated all 14 sensors are part of the xAP data reported. Please supply me more XML so I can verify my code - or else you will need to do this yourself.
This is 5 minutes worth BEFORE the latest update, which Ive just applied. GUI looks ok, but a couple of minutes in and Im still not seeing anything in XFX, so I'll start another debug.
That trace was way to short to be useful - it just started getting interesting and then you terminated it. You need to sent a much larger trace - nothing was looking wrong when I was reading it. Only the fact that it hadn't got a SYNC on the start of a <msg> frame yet. Until it does you won't get xAP message.
I really need to rewrite the code so it read from the open file descriptor straight into the SAX parser instead of accumulating in memory first this would solve your problem, I suspect, and make the code more efficient.
In the meantime a longer log file would be useful.
doh.. after stopping the serial schema for the Jeenodes, and allowing quite a few minutes to cycle, looks like all 14 +H channels now reporting.
Thanks for humouring me. I'll put load on all the channels tomorrow, and connect to Pachube, to test.
As these versions of their IAMS are two-way, now all we need to do is figure out if the existing RF control can switch them off. I'll also open one up tomorrow to see what's inside.
I just noticed that some of the content is garbage - I'll also log more data tomorrow.
HOWEVER, data is not updated - remains stuck with first value read. Eg. if sensor 3 is read as 30W, it stays that way forever. Same issue on all sensors.
I fed this through and I see only a single reading on the 1st fragment too. However this data looks partially corrupt and by that I mean the XML isn't well formed and its looks truncated in places.
Almost as if its hit some sort of maximum line length on capture. Are you sure about how this data was captured?
Please plug the UNIT into your PC and run PUTTY against the COM port directly and try again.
I know you are trying to help me, however I need some intelligent hands operating the equipment at the other end and doing diagnostics and examining the information for correctness, and perhaps troubleshooting before just shunting it all my way. I'm a busy man and I want to help you, however I don't have time to waste on examining data that you could have done yourself to discover what I've just told you. I literally get 2 hrs a day to work on the HAH project so the more you help me the more efficient I can be with the VERY limited time that I have.
I had a look at the file putty.log that was uploaded by andy_godber.
Unfortunately, or fortunately depending on your point of view, it contained the [useless] daily and monthly historic data that currentcost insist on in all but the earliest of their devices.
The xml 'lines' for these are exceptionaly long, and have excceded the buffer length allowed for in putty or its logger.
I haven't got my eco manager hooked up at the moment so I have edited andys file to remove the noise and leave the one minute data ( there is about an hours worth) in the hope that this will move things along
( I am assuming you must have had the oven on to cook dinner Andy ;) !)
"I am assuming you must have had the oven on to cook dinner Andy"
Part of the reason for my 'current' focus on getting accurate energy usage is an apparent discrepancy between the CC Optismart and clamp devices. Allegedly, the Optismart gives the most accurate reading due to reading pulses on the meter.
As you have observed, the power usage in the XML seems quite high - this was captured with a clamp. The Optismart readings are about half of that, but vary proportionally with total power; this would suggest variance due to Power Factor. However, Im not entirely convinced that the clamp readings are NOT wrong (excuse the double negatives), as they are more in line with the bills I used to get (before solar).
It may be that the Imp/KWh setting on the meter has been changed from the default/printed 1000 (which I have set on the EnviR). Does anyone know if this is possible (by the Electricity company/installer?).
Im just waiting for a couple of components to complete the OpenEnergyMonitor version which uses an AC/AC adaptor to calculate true PF, and hopefully will get a true reading of the power Im using!
BTW, I stayed in H-L-C for about a year as part of my Industrial Year/sandwich course about 25 years ago....
Which clamp are you using for the CC whole house? I found that replacing the rather large clamp with a later smaller one which was at right angle to the live cable and a tighter fit gave me 'better' readings
You should be able to check the setting of the counter on your bill, if not specifically mention you should be able to see the calculation/conversrion
Be aware that the CC units cannot be considered to be fiscal meters, but should at least "parallel" the metered values
I am unconvinced that Power Factor has any significant effect on the readings in a normal domestic property, and while I admire those people who post on the net who seem to be able to get their electricity down to low hundreds of of watts, I am sure they either are not real or that they lead sad lives on their own in a very small bedsit ;) ... My background wattage is 750 and I have replaced all incandescent bulbs with energy saving or LED ones, etc. I am quite pleased with that.
I see that you have also had a go at water meter monitoring - I need to attack this as well. The Openenergy emonTx and nanodes use the RFM12 tranceivers as well, not sure about the frequency, so would or could be made compatible with the HAH's Jeenode/HAHnode subsystem
Ive had both the small and large clamps with a CC and the SeedStudio SCT-013-000 on an OpenEnergyMonitor. Im aware that the CC are not fiscal, but the difference from the Opti is dramatic, as the following screenshots show (Opti, CC Clamp, OEM Clamp). The dip in power at about 9:30 is when solar PV kicks in - its a nice day for generating!
According to the Opti, my background usage overnight is about 550W (I know there are some servers and CCTV kit taking about 330W, plus fridge and standby kit etc), so this probably isnt far off. The clamps report it at around 900. The difference rises dramatically (proportionally?) when more power is used when we get up in the morning.
I cant see that the bill indicates what the Imp/Kwh setting is, but the calc seems to be 1:1 i.e its a modern meter showing KwH - the bill says units used and the KwH split is the same value. Ive started taking daily manual readings!
The OEM Tx are 433Mhz, but can be made to run at 868Mhz - Im going to look at getting these into HAH...eventually.
I'm not too familiar with the PF etc that is under discussion but for what it's worth, I don't consider myself to lead a "sad life in a very small bedsit" (actually a four bed shack shared with 3 other people) but my background useage at night seems to vary between about 100 and 350 watts presumably depending on the activity of fridge/freezer etc. Pachube feed 36589 stream 2.
Evening useage with lights/computers/TV's on, seems to be in the 500-900 watt range except when boiling kettles etc.
My bill from EDF for period Jul-Nov 2011 (just before my PV went in) shows an average daily usage of 25Kwh = 9000 KwH per year which would put me in the VERy HIGH group.
I do work at home at lot, but am not sure how accurate the billing was (is), hence the focus.
Pachube feed 34606/1 shows the whole house usage, using an optismart. Feed 8 is using the clamp (choose a month view for both, as feed 8 has been swapped recently).
OK so I have to admit to being a little bit sad! My long term electricity use over the last decade in a largish 4 bed place with up to 5 people is within a whisker of 14KwH per day before the installation of PV. That's just a little over 5000KwH per year. More recently (after low energy light bulbs etc.) the annual average has dropped to about 12.9 KwH per day.
25KwH per day seems quite high unless you are using some high wattage devices on a regular basis.
I live in a 4bed with a stay at home wife and two kids who do not know what a light switch is for. we regularly use dishwasher, dryer and washing machine. our typical usage is 12-18kwh. Think we did peak at 25 once but this was exceptional. 25ave sound really high. Pachube 21773
I expect that, like my kids, they know half of what a light switch is for. That'll be switching a light ON! It's the other bit that seems to be beyond them. Grrrrrrrrrr!
Well, Ive identified the likely discrepancy between the Optismart and clamp readings.
I had convinced myself that the Opti couldnt be blamed, as another on the PV inverter reported, almost perfectly, that reported by the inverter itself.
However, on revisiting the Envir DATA channel (the one where you define the Imp/KwH), the number was set at 1600, rather than the default (and needed for my meter) 1000.
Ive now corrected this, but am expecting to see usage closer to the clamp, at around 22Kwh per day :(
I'll monitor for a day or so.
If true, it does mean Ive got a spurious 700W or so being used, to identify and reduce.
Member for
16 years 10 monthsCan do
I've added minimal support for the the <chH> channel.
[inf][tx.c:19:xapSend] send
xap-header
{
v=12
hop=1
uid=FF00DC01
class=xAPBSC.event
source=dbzoo.livebox.CurrentCost:ch.H
}
input.state
{
state=on
text=788
displaytext=788 Watts
}
Update to the BETA and let me know how that goes. You need to configure as a CC128 unit.
Member for
15 years 2 monthsRight Beta?
Thanks - can I just verify Ive picked up the right beta - Build: 302.5/2.4 ?
There's nothing appearing in xFx Message Viwer other than the CurrentCost hearbeat.....
Member for
16 years 10 monthsYup right build
I simply piped your first XML fragment in as a file to xap-currentcost and I managed to get a result, this is how I tested as I don't have an EDF unit (nor a livebox for that matter). Do make sure you have configured a CC128 unit otherwise the baud rate will not be right and you'll get nothing.
If you want to run this in debug mode -d 6 and send me some logs (as a ZIP attachment please) I can take a look at why data is not being parsed for you.
This note pertains to the RAW XML from the EDF Unit: If you capture a decent interval of data (say 5 minutes worth) and compress and ZIP attach I will have a better sample of data to test and work with. A single fragment QA does not make.
I can then fully support sensors and the like and make web page amendment to integrate this unit more natively.
Brett
Member for
15 years 2 monthsNo serial in daemon
Hmm, not sure the daemon is picking it up (serial data appearing in microcom ok). Havent bothered zipping, as nothing too informative appearing:
# xap-currentcost -d 6 -s /dev/ttyUSB0 -i br0
Current Cost Connector for xAP v12
Copyright (C) DBzoo, 2009-2010
[inf][init.c:117:discoverBroadcastNetwork] br0: address 192.168.1.26
[inf][init.c:129:discoverBroadcastNetwork] br0: netmask 255.255.255.0
[inf][init.c:136:discoverBroadcastNetwork] Autoconfig: xAP broadcasts on 192.168.1.255:3639
[ntc][init.c:53:discoverHub] Broadcast socket port 3639 in use
[inf][init.c:54:discoverHub] Assuming a hub is active
[ntc][init.c:63:discoverHub] Socket port 3640 in use
[ntc][init.c:63:discoverHub] Socket port 3641 in use
[ntc][init.c:63:discoverHub] Socket port 3642 in use
[ntc][init.c:63:discoverHub] Socket port 3643 in use
[ntc][init.c:63:discoverHub] Socket port 3644 in use
[inf][init.c:65:discoverHub] Discovered port 3645
[inf][currentcost.c:453:setupXap] Selecting CC128 model
[inf][tx.c:19:xapSend] send
xap-hbeat
{
v=12
hop=1
uid=FF00DC00
class=xap-hbeat.alive
source=dbzoo.livebox.CurrentCost
interval=60
port=3645
pid=192.168.1.26:195
}
[inf][tx.c:19:xapSend] send
xap-hbeat
{
v=12
hop=1
uid=FF00DC00
class=xap-hbeat.alive
source=dbzoo.livebox.CurrentCost
interval=60
port=3645
pid=192.168.1.26:195
}
[inf][tx.c:19:xapSend] send
xap-hbeat
{
v=12
hop=1
uid=FF00DC00
class=xap-hbeat.alive
source=dbzoo.livebox.CurrentCost
interval=60
port=3645
pid=192.168.1.26:195
}
[inf][tx.c:19:xapSend] send
xap-hbeat
{
v=12
hop=1
uid=FF00DC00
class=xap-hbeat.alive
source=dbzoo.livebox.CurrentCost
interval=60
port=3645
pid=192.168.1.26:195
}
[inf][tx.c:19:xapSend] send
xap-hbeat
{
v=12
hop=1
uid=FF00DC00
class=xap-hbeat.alive
source=dbzoo.livebox.CurrentCost
interval=60
port=3645
pid=192.168.1.26:195
}
Member for
16 years 10 monthsThat explain why you are seeing nothing
Well that explain why you are seeing nothing! No data is coming into the process.
Drop down to Level 7 and you might see something, the serial input data is only displayed at these deeper levels of debugging, but you are going to get a LOT more information too.
Brett
Member for
16 years 10 monthsI'm working in the DARK here.
I've push a new BETA 302.6 which now has native support for the EDF unit. You will need to select this from the WEB GUI before things will work correctly. I've also incorporated all 14 sensors are part of the xAP data reported. Please supply me more XML so I can verify my code - or else you will need to do this yourself.
Brett
Member for
15 years 2 monthsLogs
This is 5 minutes worth BEFORE the latest update, which Ive just applied. GUI looks ok, but a couple of minutes in and Im still not seeing anything in XFX, so I'll start another debug.
Member for
16 years 10 monthsJust not enough
That trace was way to short to be useful - it just started getting interesting and then you terminated it. You need to sent a much larger trace - nothing was looking wrong when I was reading it. Only the fact that it hadn't got a SYNC on the start of a <msg> frame yet. Until it does you won't get xAP message.
I really need to rewrite the code so it read from the open file descriptor straight into the SAX parser instead of accumulating in memory first this would solve your problem, I suspect, and make the code more efficient.
In the meantime a longer log file would be useful.
Brett
Member for
15 years 2 monthsMore debug
Here is the debug after the update.
Member for
16 years 10 monthsWTF ? You have screwed
WTF ? You have screwed up.... You are feeding the CurrentCOST data into xap-serial !?
Double/Triple check what you are running and how you have configured your system.
xap-header
{
v=12
hop=1
uid=FF00D500
class=Serial.Comms
source=dbzoo.livebox.Serial
}
Serial.Received
{
port=/dev/ttyUSB0
data=<msg><src>EMC2-v1.79</src><date>14/01/2012</date><time>18:42:00</time><d14m01><kwh>23</kwh></d00m01><d15m01><kwh>0</kwh></d01m01
}
Brett
Member for
15 years 2 monthsJob's a good-un
doh.. after stopping the serial schema for the Jeenodes, and allowing quite a few minutes to cycle, looks like all 14 +H channels now reporting.
Thanks for humouring me. I'll put load on all the channels tomorrow, and connect to Pachube, to test.
As these versions of their IAMS are two-way, now all we need to do is figure out if the existing RF control can switch them off. I'll also open one up tomorrow to see what's inside.
I just noticed that some of the content is garbage - I'll also log more data tomorrow.
xap-header
{
v=12
hop=1
uid=FF00DC01
class=xAPBSC.info
source=dbzoo.livebox.CurrentCost:ch.H
}
input.state
{
state=on
text=*��`*�X
displaytext=0 Watts
}
Member for
16 years 10 monthsLets try this
Well this is why we call it BETA. Lets try with 302.7 - I think I know the problem.
If that doesn't fix it then you will need to supply an XML data feed collected from the device , this will be my last post for quite a bit.
Brett
Member for
15 years 2 monthsSome progress
All feeds now eventually appear - looking good.
HOWEVER, data is not updated - remains stuck with first value read. Eg. if sensor 3 is read as 30W, it stays that way forever. Same issue on all sensors.
Also attached is 1 hours worth of output.
Member for
16 years 10 monthsI'm not sure how you collected that data but...
I fed this through and I see only a single reading on the 1st fragment too. However this data looks partially corrupt and by that I mean the XML isn't well formed and its looks truncated in places.
Almost as if its hit some sort of maximum line length on capture. Are you sure about how this data was captured?
Please plug the UNIT into your PC and run PUTTY against the COM port directly and try again.
Brett
Member for
16 years 10 monthsI know you are trying to help me
I know you are trying to help me, however I need some intelligent hands operating the equipment at the other end and doing diagnostics and examining the information for correctness, and perhaps troubleshooting before just shunting it all my way. I'm a busy man and I want to help you, however I don't have time to waste on examining data that you could have done yourself to discover what I've just told you. I literally get 2 hrs a day to work on the HAH project so the more you help me the more efficient I can be with the VERY limited time that I have.
Brett
Member for
16 years 5 monthsAnnoying CurrentCost historic data
I had a look at the file putty.log that was uploaded by andy_godber.
Unfortunately, or fortunately depending on your point of view, it contained the [useless] daily and monthly historic data that currentcost insist on in all but the earliest of their devices.
The xml 'lines' for these are exceptionaly long, and have excceded the buffer length allowed for in putty or its logger.
I haven't got my eco manager hooked up at the moment so I have edited andys file to remove the noise and leave the one minute data ( there is about an hours worth) in the hope that this will move things along
( I am assuming you must have had the oven on to cook dinner Andy ;) !)
kevint
Member for
15 years 2 monthsEnergy Usage
Thanks for helping out Kevint
"I am assuming you must have had the oven on to cook dinner Andy"
Part of the reason for my 'current' focus on getting accurate energy usage is an apparent discrepancy between the CC Optismart and clamp devices. Allegedly, the Optismart gives the most accurate reading due to reading pulses on the meter.
As you have observed, the power usage in the XML seems quite high - this was captured with a clamp. The Optismart readings are about half of that, but vary proportionally with total power; this would suggest variance due to Power Factor. However, Im not entirely convinced that the clamp readings are NOT wrong (excuse the double negatives), as they are more in line with the bills I used to get (before solar).
It may be that the Imp/KWh setting on the meter has been changed from the default/printed 1000 (which I have set on the EnviR). Does anyone know if this is possible (by the Electricity company/installer?).
Im just waiting for a couple of components to complete the OpenEnergyMonitor version which uses an AC/AC adaptor to calculate true PF, and hopefully will get a true reading of the power Im using!
BTW, I stayed in H-L-C for about a year as part of my Industrial Year/sandwich course about 25 years ago....
Member for
16 years 5 monthsSmall world - CC accuracy, water, etc
Well small world!, I was living here then.
Which clamp are you using for the CC whole house? I found that replacing the rather large clamp with a later smaller one which was at right angle to the live cable and a tighter fit gave me 'better' readings
You should be able to check the setting of the counter on your bill, if not specifically mention you should be able to see the calculation/conversrion
Be aware that the CC units cannot be considered to be fiscal meters, but should at least "parallel" the metered values
I am unconvinced that Power Factor has any significant effect on the readings in a normal domestic property, and while I admire those people who post on the net who seem to be able to get their electricity down to low hundreds of of watts, I am sure they either are not real or that they lead sad lives on their own in a very small bedsit ;) ... My background wattage is 750 and I have replaced all incandescent bulbs with energy saving or LED ones, etc. I am quite pleased with that.
I see that you have also had a go at water meter monitoring - I need to attack this as well. The Openenergy emonTx and nanodes use the RFM12 tranceivers as well, not sure about the frequency, so would or could be made compatible with the HAH's Jeenode/HAHnode subsystem
kevint
Member for
15 years 2 monthsClamps
Ive had both the small and large clamps with a CC and the SeedStudio SCT-013-000 on an OpenEnergyMonitor. Im aware that the CC are not fiscal, but the difference from the Opti is dramatic, as the following screenshots show (Opti, CC Clamp, OEM Clamp). The dip in power at about 9:30 is when solar PV kicks in - its a nice day for generating!
According to the Opti, my background usage overnight is about 550W (I know there are some servers and CCTV kit taking about 330W, plus fridge and standby kit etc), so this probably isnt far off. The clamps report it at around 900. The difference rises dramatically (proportionally?) when more power is used when we get up in the morning.
I cant see that the bill indicates what the Imp/Kwh setting is, but the calc seems to be 1:1 i.e its a modern meter showing KwH - the bill says units used and the KwH split is the same value. Ive started taking daily manual readings!
The OEM Tx are 433Mhz, but can be made to run at 868Mhz - Im going to look at getting these into HAH...eventually.
Member for
15 years 1 monthBackground useage
I'm not too familiar with the PF etc that is under discussion but for what it's worth, I don't consider myself to lead a "sad life in a very small bedsit" (actually a four bed shack shared with 3 other people) but my background useage at night seems to vary between about 100 and 350 watts presumably depending on the activity of fridge/freezer etc. Pachube feed 36589 stream 2.
Evening useage with lights/computers/TV's on, seems to be in the 500-900 watt range except when boiling kettles etc.
Member for
14 years 11 monthsAverage according to Ofgem
As you can see, the typical consumption figure for gas has
decreased. But the figure for electricity has stayed the same.
For households on a time of use tariff, such as Economy 7,
the typical medium consumption figure for electricity is 5,000
kWh, compared to the old value of 6,600 kWh.
Customers may choose to be on a time of use tariff because
it fits their pattern of energy consumption. Some households
on a time of use tariff may have a higher consumption
of electricity than those on standard tariffs because they use
electricity to heat their home or provide hot water.
Typical low consumption values:
Gas - 11,000 kWh,
Electricity - 2,100 kWh.
Typical high consumption values:
Gas - 23,000 kWh, Electricity - 5,100 kWh
Gas Electricity
New typical medium consumption figures 16,500 kWh 3,300 kWh
Old typical medium consumption figures 20,500 kWh 3,300 kWh
http://www.ofgem.gov.uk/Media/FactSheets/Documents1/domestic%20energy%20consump%20fig%20FS.pdf
http://en.wikipedia.org/wiki/List_of_countries_by_electricity_consumption
Member for
16 years 5 monthsAlways good to provoke a reaction
Always good to provoke a reaction ;) and nothing personal of course.
Interesting to compare similar countrys in the
http://en.wikipedia.org/wiki/List_of_countries_by_electricity_consumption tables
particiularly that UK average is lower than EU average, and France and Germany particularly.
Obviously some of the differences are due to climate, infrasturcture, and consumer sophistication, but local pricing must also be a factor
kevint
Member for
15 years 2 monthsBills/accuracy
My bill from EDF for period Jul-Nov 2011 (just before my PV went in) shows an average daily usage of 25Kwh = 9000 KwH per year which would put me in the VERy HIGH group.
I do work at home at lot, but am not sure how accurate the billing was (is), hence the focus.
Pachube feed 34606/1 shows the whole house usage, using an optismart. Feed 8 is using the clamp (choose a month view for both, as feed 8 has been swapped recently).
Member for
15 years 1 monthLong term
OK so I have to admit to being a little bit sad! My long term electricity use over the last decade in a largish 4 bed place with up to 5 people is within a whisker of 14KwH per day before the installation of PV. That's just a little over 5000KwH per year. More recently (after low energy light bulbs etc.) the annual average has dropped to about 12.9 KwH per day.
25KwH per day seems quite high unless you are using some high wattage devices on a regular basis.
Member for
15 years 7 monthsI agree
I live in a 4bed with a stay at home wife and two kids who do not know what a light switch is for. we regularly use dishwasher, dryer and washing machine. our typical usage is 12-18kwh. Think we did peak at 25 once but this was exceptional. 25ave sound really high. Pachube 21773
Member for
15 years 1 monthWhat's a light switch for
I expect that, like my kids, they know half of what a light switch is for. That'll be switching a light ON! It's the other bit that seems to be beyond them. Grrrrrrrrrr!
Member for
16 years 5 monthsNot a new problem I t hink
Not a new problem I think, and not restricted to children..
Now I realise why the X10 system had a command "all lights off" !
kevint
Member for
15 years 2 monthsAccuracy issues resolved but worst fears realised??
Well, Ive identified the likely discrepancy between the Optismart and clamp readings.
I had convinced myself that the Opti couldnt be blamed, as another on the PV inverter reported, almost perfectly, that reported by the inverter itself.
However, on revisiting the Envir DATA channel (the one where you define the Imp/KwH), the number was set at 1600, rather than the default (and needed for my meter) 1000.
Ive now corrected this, but am expecting to see usage closer to the clamp, at around 22Kwh per day :(
I'll monitor for a day or so.
If true, it does mean Ive got a spurious 700W or so being used, to identify and reduce.