Up and coming changes

I've been working on some changes over the xmas break for the next HAH firmware release.
One of the items is the refactoring of the /etc/xap-livebox.ini file into separate files which will be contained in the new directory /etc/xap.d/

Member for

17 years

derek

12 years 9 months ago

The thing that I really like about the Livebox platform is that, unlike the Pi, when you buy a Livebox you get a PSU and an enclosure for your PCB/LCD thrown in. At £10 from eBay, this is hard to beat from a cost perspective.

However, things move on and there are a LOT of Pi units out there.

I'm working on a new PCB. As Brett says, it is ethernet connected. The annoying thing about the 'separate units' approach is power - these ethernet units need a wallwart. One way that we were able to keep costs down on the original PCB was to use parts that had been gathered in my basement over many years. There is still an opportunity for some of this in the new PCB.

Main change is the addition of the ethernet module. Also a twin RJ45 socket for the 1-wire bus. An unregulated 15V DC input (same as Livebox) is stepped down to give 12V for the RF Tx module and 5V for the micro. LCD is serial driven to save on I/O pins & wiring. Much of the rest is as before.

Of course, it's hard to know if anybody would want to buy such a device.

Derek.

Member for

15 years 7 months

garrydwilms

12 years 9 months ago

Quick question please Brett,

I assume that if I now do a:

 INSTALL_DIR=/ make install

After checking out the recent portable branch, I will update my PI system with all the changes made recently. Will this overwrite the previous binaries in /usr/bin? Or do I need to delete them first.

also, what about my ini files, will these get overwritten. Not sure what the install command flags do and google is confusing me. Want to be safe than sorry!

if I lose the heating for more than an hour I'm in the dog house!

Cheers,

Garry

 


Member for

13 years 7 months

bobonline19

12 years 9 months ago

Member for

13 years 7 months

bobonline19

12 years 9 months ago

Hi Derek,i feel like I'm a bit of a lurker as I'm finding very little time to put into this great project but full of ideas.I would be very tempted to go for a 5v supply for the pcb as this is becoming the standard now with most phones using USB chargers as well as the pis and similar. Could a step up circuit be used to generated the 12v required? Even a max232 can generate a few mA at 12v from 5v but there are plenty of simple step up circuits around. I could asked my colleagues for a few pointers?

Member for

14 years 5 months

mark_baldwin

12 years 9 months ago

Brett

I've run the install again using the latest update. No installation issues this time.

My only issue is that the hostname is being ignored and defaulted to arm e.g. dbzoo.arm.serial rather than dbzoo.'hostname'.serial

Mark

Member for

16 years 10 months

brett

12 years 9 months ago

Mark,

Nope the mistake is yours.  This works.  Your hostname is truely 'arm'.
Steps to update you're hostname
# vi /etc/hostname
# vi /etc/hosts
# hostname <newhost> OR # reboot.

I know this as my Beaglebone was also the hostname "arm" until I renamed it.

Brett

Member for

16 years 10 months

brett

12 years 9 months ago

Gary,

There was an issue with the direct install method in that it would overwrite ALL your .ini files.
The .DEB method does not but I missed two ini files I recently added; xap-twitter and xap-sms.
Both of these issues have been now been resolved in a commit I just made.
"svn update" to get those changes and you should be good for either installation method.

Brett

Member for

14 years 5 months

mark_baldwin

12 years 9 months ago

Brett

So it is ignoring hostname set in gui?

Ok I'll change that, thanks

Mark

Member for

16 years 10 months

brett

12 years 9 months ago

For those running a HAHCentral on their Beaglebone - I would advise not using a FTDI RS232/USB device.  For the last 4-5 days I've not been able to get this stable and it constantly crashes out.  Last night I switched to using a Prolific PL2303 cable as supplied in the shop and now its rock solid.

These are some of the strange errors that you will see being throwing in the kernel logs.

[75083.502064] ftdi_sio ttyUSB0: failed to get modem status: -19
[75083.509391] ftdi_sio ttyUSB0: ftdi_set_termios FAILED to set databits/stopbits/parity
[75083.518264] ftdi_sio ttyUSB0: ftdi_set_termios urb failed to set baudrate
[75083.525839] ftdi_sio ttyUSB0: urb failed to clear flow control
[75083.532060] ftdi_sio ttyUSB0: usb_serial_generic_submit_read_urb - usb_submit_urb failed: -19

More errors

[   24.222174] init: plymouth-splash main process (879) terminated with status 2
[   25.481942] init: plymouth-stop pre-start process (926) terminated with status 1
[  772.175976] usb 1-1: xap-serial timed out on ep0out len=0/0
[  773.175980] usb 1-1: xap-serial timed out on ep0out len=0/0
[  773.176047] ftdi_sio ttyUSB0: ftdi_set_termios FAILED to set databits/stopbits/parity
[  774.185886] usb 1-1: xap-serial timed out on ep0out len=0/0
[  774.185946] ftdi_sio ttyUSB0: ftdi_set_termios urb failed to set baudrate
[  779.194112] usb 1-1: xap-serial timed out on ep0out len=0/0
[  784.193778] usb 1-1: xap-serial timed out on ep0out len=0/0
[  784.193847] ftdi_sio ttyUSB0: urb failed to clear flow control
[  789.201975] usb 1-1: xap-serial timed out on ep0out len=0/0
[  794.201779] usb 1-1: xap-serial timed out on ep0out len=0/0
[  795.201780] usb 1-1: xap-serial timed out on ep0out len=0/0
[  795.201850] ftdi_sio ttyUSB0: ftdi_set_termios urb failed to set baudrate
[  800.209624] usb 1-1: xap-serial timed out on ep0out len=0/0
[  800.209692] ftdi_sio ttyUSB0: urb failed to set to rts/cts flow control

and more.

[ 3275.413814] ftdi_sio ttyUSB0: failed to get modem status: -110
[ 3280.421668] usb 1-1: xap-serial timed out on ep0in len=0/2
[ 3280.421738] ftdi_sio ttyUSB0: failed to get modem status: -110
[ 3285.429518] usb 1-1: xap-serial timed out on ep0in len=0/2
[ 3285.429588] ftdi_sio ttyUSB0: failed to get modem status: -110
[ 3290.443283] usb 1-1: xap-serial timed out on ep0in len=0/2
[ 3290.443352] ftdi_sio ttyUSB0: failed to get modem status: -110

Kernel FTDI drivers are buggy and I would advise not using this chipset.

Brett

Member for

16 years 6 months

kevin

12 years 9 months ago

Wallwarts.. a very approriate name....

It's nice that there are increasing number of POE devices appearing, and I have bought POE switches when upgrading but it is still quite costly (component wise) to include the 802.3af electronics onboard.   In the meantime I mostly use those small inline POE extractors to avoid wallwart congestion.

Member for

14 years 5 months

mark_baldwin

12 years 8 months ago

Brett

I'm having issues with xap filters not picking anything up with 591 (it started on an earlier release but can't remember which)

I can see the serial data in xfx viewer and have edited the filters to include capitalisation where it shows in xfx but still no data 

Here is an example filter

f = xap.Filter()

    f:add("xap-header","source","dbzoo.beaglebone.Serial")

    f:add("Serial.Received","port",port)

    f:callback(update)

the line to read the data

local data = frame:getValue("Serial.Received","data")

and the data it should pick up.

xap-header

{

v=12

hop=1

uid=FF00D500

class=Serial.Comms

source=dbzoo.beaglebone.Serial

}

Serial.Received

{

port=/dev/ttyUSB0

data=CC:Tx:4073:106:END

 

}

I have also included the lua script in full.

It works fine on the arm port you did that runs under Angstrom.

If there is any more info you need, just let me know.

No rush, it's on the BB dev board, not my HAH

Cheers

Mark

Attachments
CCApplet.lua.txt (2.33 KB)

Member for

16 years 10 months

brett

12 years 8 months ago

The only different code wise from the Angstrom distro via the new portable/Ubuntu distro is that I use the penlight library that is packaged with Ubuntu.   I just did a diff are compared it to the one I shipped with the livebox and guess what?  Its got bugs I know cause problems. bugged I raised with the penlight developer and got fixed.

Ubuntu are shipping a buggy penlight library.  I ship the latest as part of the portable distribution and insist the ubuntu version is removed.
I'm not saying that is the problem but there is no other code change between the two - so I'm guessing its a good place to start.

As a workaround can you try removing the penlight library from you BB and copy over the one from you're livebox.
As the HAH depends on lua-penlight you can't uninstall it so just replace it.

brett@beaglebone:$ sudo rm -rf /usr/share/lua/5.1/pl
brett@beaglebone:$ sudo scp -r root@livebox:/usr/share/lua/5.1/pl /usr/share/lua/5.1

Brett

Member for

15 years 7 months

garrydwilms

12 years 8 months ago

Hi Brett,

I've noticed that the Plugboard log is not getting populated on my pi?  I have checked the init.d/xap script and >/var/log/xap-plugboard.log 2>&1 is present.

Is this an issue on the pi only? Does the bone report log correctly?

 

thanks,

Garry


Member for

14 years 5 months

mark_baldwin

12 years 8 months ago

Brett

That seems to have cured it. 

Cheers

Mark

Member for

14 years 5 months

mark_baldwin

12 years 8 months ago

Garry

Do you mean there is nothing in the log or no errors?

2>&1 means send stderr to file (apparently) so the log should only be populated on errors

Member for

15 years 7 months

garrydwilms

12 years 8 months ago

Hi mark,

on the livebox you used to get a list of applets being loaded into the log. Also if you put print command into your scripts for debugging they would be printed to the file. I get nothing on my pi. Not tried to see if errors are logged but the stdout doesn't seem to be logged?

 

cheers

Garry

Member for

16 years 10 months

brett

12 years 8 months ago

Gary,

Welcome to the land of channel buffering.  Just because you redirect to a file does not mean it will appear immediately.
The content is in a memory buffer waiting to be flushed when the channel closes, or its force out.
This will happen when the process terminates or the buffer fills.
As plugboard did not open it, the shell redirects did, it has no way to force it to flush.
Your print debug statements are not effectual for realtime debugging - which is why I tend to develop applets in a more standalone mode outside of the plugboard before they are integrated.   Remember this?  Its written up on the wiki plugboard section.

Brett

Member for

15 years 7 months

garrydwilms

12 years 8 months ago

Cheers Brett, that explains it!

can run standalone in future, no probs

 

Garry

Member for

12 years 8 months

johnnyhah

12 years 8 months ago

Hi folks,

errors I get trying upgrade on my Pi

$ cd /home/pi/portable
$ svn update
$ make arm-deb
$ cd /home/pi/portable/build/
$ sudo dpkg -i hah_0.1_armhf.deb

 

(Reading database ... 86526 files and directories currently installed.)
Preparing to replace hah 0.1 (using hah_0.1_armhf.deb) ...
Unpacking replacement hah ...
dpkg: error processing hah_0.1_armhf.deb (--install):
 trying to overwrite '/usr/share/lua/5.1/pl/stringx.lua', which is also in package lua-penlight 1.0.2+htmldoc-2
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
update-rc.d: using dependency based boot sequencing
insserv: warning: script 'mathkernel' missing LSB tags and overrides
update-rc.d: using dependency based boot sequencing
insserv: warning: script 'mathkernel' missing LSB tags and overrides
Errors were encountered while processing:
 hah_0.1_armhf.deb

So this upgrade fails . .  when I look at the top of Kloned web page it still shows Build: 312/?Err? so its not 313 as reported in the /home/pi/portable/build/sysroot/etc/xap.d/build,  ?Err? is just caus I've no hardware on the pi yet.
What am I doing wrong?

any help appreciated

john

Member for

16 years 10 months

brett

12 years 8 months ago

Johnny,

The portable release is still ALPHA which means you are going to get problems like as there is no upgrade path between releases until I get some time to do some more development work on it.

The reason this is happen is because lua-penlight was removed as a dependency and included into the HAH distribution due to bugs in the lua-penlight package that is installed with apt-get.

Now how to unwind you..

As you've discovered you can't just upgrade with

dpkg -i ..... etc.

For the reasons I've just mentioned above.   You need to complete remove the HAH installation.  You won't loose any of your settings BUT I would back things up to be on the safe side. * I'm typing this haven't not tested it, as I jury rigged my system a different way, however this is the approach I would recommend you take, to make this problem go away permantly *

dpkg -r hah

Then remove lua-penlight as its no longer required.

dpkg -r lua-penlight

Then proceed with installing the HAH system again like you attempted.

The build numbers for the PORTABLE distributtion are not really relevant at this point and they won't be in step with the livebox distribution.  The ?Err? is because there is no hardware attached to the PI.  However as I said this is ALPHA and the entire kloned webserver needs to be overhauled as there are parts that a) don't make sense b) just don't work on the PI.

Brett

Member for

15 years 10 months

g7pkf

12 years 8 months ago

Been "off the grid" so to speak for a while the new changes look good, now the question's.

 

running a hah on a livebox.

is the upgrade straightforward (yes i will back up first) on livebox?

what exeactly is needed to transfer from livebox to Pi? to keep functionality.

Love Pi and have a spare.

on HAH i have pcb + jeenode + currentcost.

sorry for asking such basic questions

Also ref Pi is a SD download avalible so i can just burn an sd and get up and running fast (rather than going through a build process-and a lot of pain) i am a hardware guy not a software developer.

Member for

14 years 5 months

mark_baldwin

12 years 8 months ago

The Pi and BB versions are still at alpha stage but Garry has managed to attach his HAH pcb to the gpio. 

I'm running on livebox and BB, gradually moving stuff over. Like you I have HAH pcb, basenode (with URFRX mod) and CC attached to the livebox. The CC is almost redundant now though as I have a node running on the BB that decodes CC. I also have another node on the BB that decodes the WH1080 weather station. 

To sum up my current set up. The Livebox is used for CC, RF gasnode and roomnodes. The BB is used for CC, WH1080 and presence detection using arp to ping known wifi attached phones. Both BB and livebox publish to Xively and they both access each others xap messages using plugboard scripts. I'm using an app created in Corona for control/monitoring of the overall system.

There isn't a downloadable SD version yet and no install package either so you will have to download and build. It's not too hard once you try it a few times. I can only really comment on my BB experience though. Garry has played with the Pi.

Member for

15 years 1 month

allanayr

12 years 8 months ago

Hi Dean, welcome back,

I've got the software running on my Pi it's straight forward enough if you follow the instructions on the portable wiki. I donno if it would help but I've attached a zip version of the .deb as compiled on my Pi which you might just be able to install as a package if you are using the raspbian wheezy distro.

 

Allan

Attachments
hah_0.1_armhf.zip (984.51 KB)

Member for

15 years 7 months

garrydwilms

12 years 8 months ago

Hi dean, maybe give Allan's zip a go, otherwise I am planning on doing a virgin pi install for my brother soon (next week) so can take an image before I customise if you want?

As for PCB, it fits on the pi gpio easily, just a breakout board of fleabay and a change in serial port.

as for recent changes, only work involved should be modding your scripts slighty for the new way to initialise endpoints and require modules. The ini file is now split but the update should take care of that.

to be honest though, if I were in your position I would leave your livebox running as is and start to transition to the pi on the new version, that way you can play without upsetting your existing setup. just move the pcb over last when everything else working.

 

Garry

Member for

15 years 10 months

g7pkf

12 years 8 months ago

in fact sure i have a spare HAH board, just need to get a micro programmed up and get this break out board you speak of. :)

Member for

14 years 5 months

mark_baldwin

12 years 7 months ago

I made the leap to 'almost' total control from the beaglebone. The only thing left to move over is the HAH pcb (RF) and iServer. I can only presume that if iServer works on the BBB then it will be faster than the livebox. 

As an aside, it's been very quiet on here recently. I know it's not winter anymore and the heating season is behind us (and the days of needing your lights on almost 24HRS) but now is the time to play and prepare for next winter ;)

Member for

13 years 7 months

bobonline19

12 years 7 months ago

Once I have moved to the Pi I was hoping to look at Node Red as a way of visualizing the plugboard but Martyn got there first.

http://blog.aceshigh.net/2014/04/the-futures-bright-the-futures-red-node-red/

 

I was also thinking that the Spark.IO boards looked interesting for separating the hardware from the main processor but Brett got there first.

/content/standalone-urf-tx-unit#comment-3596

https://www.spark.io/

Member for

14 years 5 months

mark_baldwin

12 years 7 months ago

Brett we need a trap for data collisions on the BB

I have traps set up for false data on the Current Cost and WH1080 code but it seems jeenode.lua fails if unexpected data arrives.

This caused plugboard to stop

lua: /usr/share/lua/5.1/xap/init.lua:436: /usr/share/lua/5.1/xap/jeenode.lua:51: invalid value (nil) at index 2 in table for 'concat'

xAP Message being processed.

xap-header

{

uid=FF00D500

source=dbzoo.beaglebone.Serial

hop=1

class=Serial.Comms

v=12

}

serial.received

{

data=OK 2 89 0CC:Tx: 193 0

port=/dev/ttyUSB2

 

}

Member for

16 years 10 months

brett

12 years 7 months ago

Mark,

OK - but my question is how can this happen in the first place?  This looks self induced by modifying the HAH base node code and allowing this to happen.   Perhaps the jeenode.lua should be robust to this but its not something everybody is going to see.

Brett

Member for

14 years 5 months

mark_baldwin

12 years 7 months ago

Brett

The error seems to be caused by 2 serial messages colliding. Again the question is why would this happen if the data is on different serial ports. So I don't think is has anything to do with the basenode mod or any problem in jeenode.lua.

It just shouldn't happen but it is.

I've just had another, this time it was the CCApplet that stopped not jeenode.lua

There error message shows data that should be on ttyUSB0 mixed with data from a roomnode that 'should' come in on ttyUSB2

serial.received

{

data=CC:Tx:39OK 685:226 164 0 1:END

port=/dev/ttyUSB0

}

 

Mark

Member for

16 years 10 months

brett

12 years 7 months ago

Mark,

Perhaps there is a buffering issue in the xap-serial code.
This component handles the inputs from both serial ports at the same time and a malfunction in here could do this.
Maybe its not clearing its buffer before servicing each serial port.
You could try enabling debug on this component and see if you see the data collsion happen.
In the meantime I'll will examine the code to see if this could happen.

UPDATE: looking at the code this this can indeed happen.  My implementation is nieve in that is uses a common buffer for servicing ALL serial port messages.  If an incomplete read happens, that is a read not to the end of the line, and another serial port is read this gets interposed into the buffer.   On the low clock speed of the MIPS (livebox) the data could feed the chip almost as fast as it could read so this didn't happen.   On the higher clock speeds the cpu gets a rest whilst waiting for different serial ports to present so it goes around servicing other ports causing corruption.  DAMN.

I need to rework this logic now I understand how this is happening.  Nice catch.

Brett

Member for

14 years 5 months

mark_baldwin

12 years 7 months ago

Brett

r595 under test now. No issues so far.

Mark

EDIT: Still running fine after 26hrs. Good work Brett

Member for

16 years 10 months

brett

12 years 7 months ago

Mark,
  We might call this one solved.  I'll leave it for another week then roll the same change into the livebox code base.
Brett

Member for

14 years 5 months

mark_baldwin

12 years 7 months ago

Brett

Still running fine.

Mark

Member for

14 years 5 months

mark_baldwin

12 years 5 months ago

I thought I'd have a look at setting up a serial port on the GPIO pins for the HAH PCB. A bit of a minefield but making a little progress. I'll post back as soon as I have made a little more progress.

Member for

14 years 5 months

mark_baldwin

12 years 5 months ago

Be careful what you do if you edit the optargs line in uEnv.txt on the BBB. Especially if part of it disables the eMMC.

I did have a backup but that didn't work :(

All back up and running now but I'm now running from the SD card (much easier to work with)

Member for

14 years 5 months

mark_baldwin

12 years 4 months ago

Brett

I want to connect to UART4 on the BBB so I can use the connections on P9. I have got the port working and can ' see' the HAH AVR on the port but what file do I modify (if any) to allow control of my RF devices?

Cheers

Mark

Member for

15 years 7 months

garrydwilms

12 years 4 months ago

Hi Mark,

 

i remember Brett making changes to xap-live box to allow different serial port to be used. Is it not just a case of amending the following in /etc/xap.d/xap-livebox.ini?

 

[livebox]
enable=0
#port=/dev/ttyS0

 

The pi for example uses /dev/AMA0 on the gpio pins.

 

Garry.

 

EDIT: just remembered I also had to 'hack' the etc/init.d/xap script to send a '+++' command to the board on startup. E.g.

start_livebox() {
INI="/etc/xap.d/xap-livebox.ini"

if [ `iniget $INI livebox enable 0` -eq 1 ] ; then

echo +++ >/dev/ttyAMA0

/usr/bin/xap-livebox $MYDEV >/var/log/xap-livebox.log 2>&1 &
fi
}


Member for

14 years 5 months

mark_baldwin

12 years 4 months ago

Hi Garry

I'm using the GPIO P9 11 & 13 for TX and RX using UART4 so the port would be /dev/ttyO4

root@beaglebone:~# ll /dev/ttyO*

crw-rw---- 1 root tty     248, 0 Jul  1 20:41 /dev/ttyO0

crw-rw---- 1 root dialout 248, 4 Jul  1 20:41 /dev/ttyO4

ttyO0 is used for the terminal, I Suppose I could use that

I've set up the files as required and as soon as SWMBO has done some soldering for me tomorrow, I'll let you know how it\s gone.

I know we can edit the .c files and do a build, I had to for my Xively as I had more than 32

Cheers 

Mark

Member for

16 years 10 months

brett

12 years 4 months ago

I had to check the source code for supplying a serial port to xap-livebox as I could not recall if it would honour the INI file.
You have to supply it on the command line, if you don't it defaults to /dev/ttyS0,  so override like this:

xap-livebox -s /dev/ttyACM0

I guess I could make it use an INI file this way the init.d/xap file does not need tweaking.  Then again as Gary said you still need to echo +++ out the serial port to hack past what the livebox was doing.    I know I can kill two birds here.

1. I will make xap-livebox pick up port= from its /etc/xap.d/xap-livebox.ini file.
2. I'll add another param to get it to echo +++ for you !

port=/dev/ttyO4
initserial=+++

A variable called initserial and it will pick this up and whever is there it will send down the serial port on startup.
Do it?

UPDATE - the portable branch does handle port= as an ini setting.  I was looking at the trunk when I checked !

Brett

Member for

14 years 5 months

mark_baldwin

12 years 4 months ago

All moved to the BBB now