Twitter Service Stopped working

Hi,

This post is more for info, as I have been unable to recreate this.

I was playing around with the twitter feature from my phone. Using ubertwitter to post commands to the relays. I did not get any response. I had got responses in the morning but not in the afternoon. 

By response I mean deleting of the command tweet and a tweet from the box, which says "Relay x is off".

When I eventually got home tonight, using the xfx viewer there was no twitter present, in the item list and no heart beat.

Member for

16 years 10 months

brett

15 years 9 months ago

If the twitter process isn't showing up in the xfxviewr then it sounds like it has crashed.  Unfortunately due to space limitiations on the HAH all core dumps are prevented and even if they where enabled all the binaries have their symbolic debug tables stripped (due to limited flash space).   So there is not much you can provide except keeping you eye on it and seeing if its reproducable.   I haven't had any other crash reports.

Brett

Member for

15 years 10 months

danward79

15 years 9 months ago

Ok thanks Brett. If it happens again I might be able to set up something via the Plugboard scripting.

Member for

15 years 10 months

danward79

15 years 9 months ago

Hi Brett,

It happened again yesterday while I was at work. During this time the service was not used at all.

So I scheduled a auto reboot at 0100 hrs everyday, hoping that would be a work around. It rebooted this morning at 0100 hrs and when I got up at 0545 and checked it, it was not responding to twitter.

I rebooted the box at 0605 and it has been ok till now 1100 hrs.

No one else has posted suggesting it is an issue. I wonder if they are using twitter much and if they have any reboots etc scheduled.

In terms of capturing this, like you say it is difficult.

I was wondering about setting yo a timer script which is reset by the heartbeat, logging this on google calendar when it is not detected for more that the time out.

Is there anyway I could setup a script to record a text file to a usb stick?

Do you have any ideas?

Cheers

 

Dan

Member for

16 years 10 months

brett

15 years 9 months ago

The logs where uber useful I have a fix for this problem (I think).  The easiest way to test this is for me just to push a new build and see what happens rather then waiting for twitter to time out its API again, which could take a while.  I'll leave my system running twitter with debug on as well.

Unfortunately I've been working on some other changes in the mainline trunk that I have yet to regression test using the older M8 firmware.  So if I push this it (might) break everybody's HAH system.  I'm using the yet to be released 328 chip and different AVR firmware you see.  *sigh* the delimer.

Tell you what I'll push this as a BETA and you can test this out and make sure my other changes are ok too.

/etc_ro_fs/update-dev hah-beta.dbzoo.com

Brett

Member for

15 years 10 months

danward79

15 years 9 months ago

Hi,

Excellent. I'll happily test a beta version for you, my system is not doing anything useful at the moment. So there is zero risk.

I have updated and will keep my eyes open. Thanks again.

 

Dan

Member for

15 years 10 months

danward79

15 years 9 months ago

Hi Brett,

Been running this all day so far I have not seen a crash. I have removed the scheduled reboot for tonight to give it a chance.

Looking Good!

Thanks

Dan

Member for

16 years 10 months

brett

15 years 9 months ago

Well let it run over the weekend before declaring this a success if you don't notice any other defects then I suspect the other changes I did are compatible with your AVR firmware.  This is good, they should have been.  Once its in the clear I'll push a new build for all.

Brett

Member for

15 years 10 months

danward79

15 years 9 months ago

Hi,

I hate to say it but it died again today. I have not managed to log it yet thou.

I'll try and log it tonight / tomorrow.

Dan

Member for

16 years 10 months

brett

15 years 9 months ago

I'm going to have to run the twitter in GDB overnight and try and catch the error.  It might take me a little to figure out now. *sigh*

Member for

15 years 10 months

danward79

15 years 9 months ago

On a positive not at least you are seeing it too... ;-)

Let me know if you want me to capture the logs too.

 

Thanks

Dan

Member for

17 years

derek

15 years 9 months ago

... since my last reboot. xap-twitter still running. No tweets handled, but it's there.

Derek. 

Member for

15 years 10 months

danward79

15 years 9 months ago

Hi Brett,

Did you manage to find the problem?

Dan

Member for

16 years 10 months

brett

15 years 9 months ago

I ran it in GDB and the program is not crashing per se its actually doing a clean exit.  Which is worse a crash is easier to find.

I think there is something funky happening in the Curl library I even ran an strace but it didn't tell me anything I didn't already know.... so now I'm going to have to mock up an apache server that generates a 503 return code and run this snippet locally stepping through the code.  As you can tell this is going to take time - rather damn annoying actually.

Brett

Member for

16 years 10 months

brett

15 years 9 months ago

I'm pretty sure I've nail this problem update to beta 279.3 let it run for a few days and let me know how you get on.

PS: This build has the new LUA engine so don't just upgrade from the mainline build unless you are prepared to rewrite any LUA scripts you have.


Member for

15 years 10 months

danward79

15 years 9 months ago

Hi Brett,

I've just uploaded it to the box, all my scripts are converted to the new lua stuff anyhow.

I'll let you know how I get on.

 

Dan

Member for

15 years 10 months

danward79

15 years 9 months ago

Hi Brett,

It has lasted till today from the 17th. So a vast improvement, unfortunately just not a total fix.

Dan

Member for

16 years 10 months

brett

15 years 9 months ago

I'll run my twitter in debug mode and see if something turns up - can you run yours in debug mode too?

One of us might catch something useful.  I know i fixed the 503 return code issue so not sure what this might be.

Member for

15 years 10 months

danward79

15 years 8 months ago

Hi, Just to let you know I am still trying to capture a crash.

I did have it crash after two days of logging but the data was lost by swmbo.... I have since not been able to get a crash! The log fine is over 140Mb.

Have you spotted anymore your end?

Dan

Member for

16 years 10 months

brett

15 years 8 months ago

I ran my capture for a while but I did not see any issue.   I had a power outage and my server rebooted since then I haven't bothered to set it back up again.  Having said that I've never seen twitter down since my fix.

Member for

15 years 10 months

danward79

15 years 8 months ago

I must admit I am on the verge of giving up on the logging for now. I might right a script to restart it if it crashes.