Off-cloud research #4

jilleb · 2020-12-08T14:12:18Z

@guino, I hope you don't mind me creating a seperate issue, but I think it helps to keep the other topics on-topic and easier to read.

So let's discuss here what's what when it comes to off-cloud/offline usage of the doorbell. Currently it uses the Tuya cloud, which means the following, according to @m11tch research

when internet connectivity is blocked:

you can not view recordings from tuya app, so seems to be streaming directly from SD upon request.

RTSP is not available. (connection refused) (will check logs later)

snap.cgi and mjpeg.cgi work fine

button press does make the "ding dong" sound, but no mqtt message is sent (will check logs later)

Camera still switches to night mode if dark

m11tch · 2020-12-08T14:15:43Z

@jilleb I was thinking the same as we were taking over the other thread quite a bit :D
Doing some more research later, will update this post with new findings.

Current status:

offline mode is possible(online mode will also work, so use your firewall to block access)
RTSP stream works, no more crashes.
snap.cgi and mjpeg.cgi work fine
logparser can be used to send mqtt messages on motion / button press
RTSP over TCP consumes much less CPU compared to RTSP over UDP (About 10-20 VS 95) (apps like VLC and mplayer seem to default to UDP) so it is recommend to use RTSP over TCP instead. RTSP over TCP also seems alot more stable. (running without problems for over 12 hours.)

Not possible:

2 way audio

offline mode ppsapp:
https://github.com/guino/BazzDoorbell/files/5672301/ppsapp-rtsp4.zip
NOTE: only for following device/fw version:

firmware version	hardware version	original ppsapp MD5	device
ppstrong-c51-tuya2_lcs-2.9.6.20200628	BE8S_H1_V10_433	50ad9c96c65c0e446d8b3d5c8c828957	Bell 8S (LSC Smart Connect Smart Doorbell)

Have a different ppsapp version? check out @jandy123 's write up on patching it:
#4 (comment)

logparser:
credits to @jandy123
mqtt_log_parser.zip

Modify custom.sh, when about to restart ppsapp, to something like:

# debug info from ppsapp  forwarded to output.log
  /mnt/mmc01/ppsapp  2>&1 | /mnt/mmc01/log_parser.sh /mnt/mmc01/output.log &
# no debug info  
#  /mnt/mmc01/ppsapp 2>&1 | /mnt/mmc01/log_parser.sh /dev/null &

Upload log_parser.sh and mosquitto_pub to the root of sdcard; see archive contents.
Modify log_parser.sh as needed, i.e., change mqtt server address, etc.

jilleb · 2020-12-08T14:37:57Z

dmesg output when doorbell is pressed while it's not connected to the cloud:

initS (906): drop_caches: 3
door bell  press down
 Strnio_ioctl
AcShdn,open pwr
CMT211XA_Send size: 68
 ###cfg gpio OK name:spi1_sdo
 cfg gpio ok:spi1_sdo
irq-----
send data:68
initS (906): drop_caches: 3
door bell  press up
send data:68
send data:68
send data:68
send data:68
CMT211XA package Send finished!
initS (906): drop_caches: 3
 Strnio_ioctl
AcShdn,close pwr

I use while true; do dmesg -c ; sleep 1 ; done to show the scrolling log

jilleb · 2020-12-08T15:01:13Z

I don't see motion detected in dmesg though, perhaps motion detection is happening in tuya cloud or it is part of ppsapp which is not fully started yet?

Good question.. I'm looking for it in Ghidra. Sounds like something that would possibly happen in the cloud. I'll get back on this.

guino · 2020-12-08T15:30:24Z

I have been downloading/viewing my .data files for the longest time - they play just fine in VLC when you set the demux to h264 demuxer (in advanced settings). It is kind of a pain having to switch between automatic and h264 but it works - just no audio.
I personally like having the tuya app, so I plan to keep it. Just wanted to be able to use RTSP, mjpeg and be able to backup the SD card files as they’re created - and now I can do it all.
Also I don’t record everything, only motion events and my camera doesn’t run hot at all.
@jandy123 if your camera is running hot, check if you don’t have any left over streamer-arm or other processes running in the background from testing, etc. The mjpeg support I put together is just a jerry rig, so the ideal solution is that an application should be made for that purpose (and to make it more efficient).

guino · 2020-12-08T15:32:19Z

The fact that I don’t record everything may also impact if the camera runs hot or not (haven’t tried recording everything) plus I have a 9v transformer at the house doorbell.

m11tch · 2020-12-08T15:32:53Z

I don't see motion detected in dmesg though, perhaps motion detection is happening in tuya cloud or it is part of ppsapp which is not fully started yet?

Good question.. I'm looking for it in Ghidra. Sounds like something that would possibly happen in the cloud. I'll get back on this.

maybe getting ppsapp to run propperly is enough?, i'm no expert in Ghidra 😞 maybe it is enough if we can disable the mqtt connection in ppsapp as it is just constantly trying to reconnect, each time with a longer sleep time.. and not doing much else...

[01-01 01:00:35-- TUYA Debug][mqtt_client.c:1230] mqtt fail_cnt:6 and sleeptime:4756 ms

jandy123 · 2020-12-08T15:45:35Z

@guino:

Welcome back ! Guess, different timezones, jobs, etc.
I have no other apps running, streaming etc. And this is without enabling continuous recording. Just record on event setting in the app. It's also interesting that for you, apparently, 9V AC is enough. I couldn't get it running with anything (apart from USB) but with the supplied 12V DC adapter.
If I run it from USB, then it's getting a bit warm on the opposite side of the lens. If I run it from the 12V adapter, it gets quite hot around where the wires are connected. Again, I suspect some LDO and/or diode bridge ? I.e., dropping from 12V AC to 3.3V has to happen somewhere...

jandy123 · 2020-12-08T15:47:04Z

For me rtsp streaming still works after blocking outside connection.

Edit: Also, the doorbell push button gets transformed to mqtt event using log_parser. Could you guys confirm this ?

jilleb · 2020-12-08T15:58:15Z

I'll check. Rtsp does work after blocking web access. But upon reboot it's dead.

I just restarted ppsapp from the console by accident.. got a lot of interesting output I will post later when I'm back at the pc.

guino · 2020-12-08T15:59:03Z

I would expect recording continuously would make it hotter than recording events only - have not paid attention to be honest. The other doorbell I had (EZVIZ DB1) was hot as hell compared to this one. It may require more current to run at lower voltage (power=voltage * current) so maybe a bigger low voltage transformer would work, just not sure if it’s worth messing with it when it is already working.

If you guys are going to be doing stuff offline using the logs I think we should patch the debug log function to always print the output (there’s some sort of output level setting which I haven’t tried to find yet).

Regarding mqtt are those messages encrypted or require secret keys? Cause if not you could just write a new address in ppsapp and use your own server? Just a thought. It should be possible to ‘patch’ ppsapp into thinking the connection is working but that’s a bunch more work specially to do in multiple ppsapp versions.

In my case I have a guest network that only has internet access where I put these untrusted devices , then I limit what I access with iptables/ebtables, so I don’t mind that they’re online.

m11tch · 2020-12-08T16:08:41Z

@guino connections to the tuya mqtt server are over TLS and require username/password. for internal usage this is ofc not necessary.. I will try redirecting the connection to a internal mqtt server that supports TLS to see if that is enough..

also saw ppsapp print a username and password (and more info) to be used for mqtt connection, but these did not let me connect to the tuya server.. perhaps password is a hash or encrypted?

jandy123 · 2020-12-08T16:12:01Z

@guino

It may require more current to run at lower voltage

Nope, this is not the cause for heating, since it's definitely cooler at 5V from USB. It's just some linear regulator which transforms the 12V > 3.3V that dissipates heat, I guess. But then, I didn't even open the device yet...

If you guys are going to be doing stuff offline using the logs I think we should patch the debug log function to always print the output (there’s some sort of output level setting which I haven’t tried to find yet).

As I said, I get same log output from ppsapp, so I can still see the forwarded mqtt events.

jandy123 · 2020-12-08T16:14:42Z

Also, in my case, the RTSP server seems to be created on the fly, when a client request (from LAN) is made. This works (as it should !!) without the cloud struff. I can post logs if needed.

jandy123 · 2020-12-08T16:16:20Z

@m11tch

I will try redirecting the connection to a internal mqtt server that supports TLS to see if that is enough..

This won't work. I've already looked at the communication in wireshark...

guino · 2020-12-08T16:18:48Z

It seems they have made changes from 2.9.6 to 2.9.7 in a number of aspects (ie blocking HD RTSP, changes to log functions, some logging and probably more) -- those are the things I noticed when patching some of the files.

m11tch · 2020-12-08T16:19:03Z

@m11tch

I will try redirecting the connection to a internal mqtt server that supports TLS to see if that is enough..

This won't work. I've already looked at the communication in wireshark...

I don't need the actual data, was just hoping that it would be enough to let ppsap to start up further..

@jandy123 RTSP does not seem to happen for me, what kind of request are you sending?

@guino I belive both me and jandy are running 2.9.6 though..

jandy123 · 2020-12-08T16:22:33Z

@m11tch

I belive both me and jandy are running 2.9.6 though..

Well, yes, but then we have different ppsapp...

jandy123 · 2020-12-08T16:23:59Z

As noted in the other thread by @adwiraguna, CPU usage in rtsp streaming mode gets to 100%. IS this only with blocked tuya servers ???

jandy123 · 2020-12-08T16:25:45Z

@m11tch

what kind of request are you sending?

What exactly you mean ? I just say vlc rtsp://192.168.2.179:8554

m11tch · 2020-12-08T16:27:41Z

As noted in the other thread by @adwiraguna, CPU usage in rtsp streaming mode gets to 100%. IS this only with blocked tuya servers ???

Perhaps this is why they decided to block HD rtsp streams in 2.9.7?

@jandy123 I will try to open rtsp stream again later today when connection to tuya is blocked.. will come back with the results..

jandy123 · 2020-12-08T16:42:44Z

Well, CPU usage when streaming over RTSP is not that bad, actually.

Assuming blocked tuya servers, idling is round 3-5% (reported by top). When HD rtsp streaming to one client, it goes up to about 17-20%. The process itself ppsapp does take about 100% CPU, though.

Edit: Pushing the bell button while streaming also works ;)... and even the mqtt event gets forwarded, so it's just fine.

Temperature-wise, hmmm, still too hot, in my opinion (12 V AC).

jilleb · 2020-12-08T17:06:44Z

Also, in my case, the RTSP server seems to be created on the fly, when a client request (from LAN) is made. This works (as it should !!) without the cloud struff. I can post logs if needed.

We have the same doorbell/firmware/ppsapp, but it isn't working

debug log from ppsapp:

debug.txt

The relevant part seems to be:
IPC DEFS < ENABLE_ECHO_SHOW:1 ENABLE_CHROMECAST:1 ENABLE_CLOUD_STORAGE:1 >'

if we can influence these definitions.. would be cool

jandy123 · 2020-12-08T17:13:48Z

@jilleb I now see what you mean. If start ppsapp without tuya cloud connection, indeed, rtsp streaming does not work. Seems that ppsapp gets stuck on connecting to tuya hosts:

`
[01-01 01:01:57-- TUYA Err][mqtt_client.c:1051] resolve mqtt host Fail:m2.tuyaeu.com erro:-100
[18:12:27.766 DEBUG tuya_network_manager.c:535] WIFI[0] Get IP:192.168.2.179
[18:12:28.009 DEBUG pps_net_manager_client.c:189] NETWORK KEEPALIVE
[18:12:32.773 DEBUG tuya_network_manager.c:535] WIFI[0] Get IP:192.168.2.179
[18:12:37.782 DEBUG tuya_network_manager.c:535] WIFI[0] Get IP:192.168.2.179
gethostbyname error 0

`

m11tch · 2020-12-08T17:27:07Z

Also, in my case, the RTSP server seems to be created on the fly, when a client request (from LAN) is made. This works (as it should !!) without the cloud struff. I can post logs if needed.

We have the same doorbell/firmware/ppsapp, but it isn't working

debug log from ppsapp:

debug.txt

The relevant part seems to be:
IPC DEFS < ENABLE_ECHO_SHOW:1 ENABLE_CHROMECAST:1 ENABLE_CLOUD_STORAGE:1 >'

if we can influence these definitions.. would be cool

I noticed some config files in /home/cfg, maybe we can do something with those to manipulate? although I don't see anything usefull at first glance..

jandy123 · 2020-12-08T17:27:43Z

@jilleb

I never get to

IPC DEFS < ENABLE_ECHO_SHOW:1 ENABLE_CHROMECAST:1 ENABLE_CLOUD_STORAGE:1 >

In my case, ppsapp gets stuck looping the bove message.

m11tch · 2020-12-08T17:30:08Z

@jilleb

I never get to

IPC DEFS < ENABLE_ECHO_SHOW:1 ENABLE_CHROMECAST:1 ENABLE_CLOUD_STORAGE:1 >

In my case, ppsapp gets stuck into

[01-01 01:01:57-- TUYA Err][mqtt_client.c:1051] resolve mqtt host Fail:m2.tuyaeu.com erro:-100 [18:12:27.766 DEBUG tuya_network_manager.c:535] WIFI[0] Get IP:192.168.2.179 [18:12:28.009 DEBUG pps_net_manager_client.c:189] NETWORK KEEPALIVE [18:12:32.773 DEBUG tuya_network_manager.c:535] WIFI[0] Get IP:192.168.2.179 [18:12:37.782 DEBUG tuya_network_manager.c:535] WIFI[0] Get IP:192.168.2.179 gethostbyname error 0

I think connection was not blocked during the startup of ppsapp in this debug log, can you confirm @jilleb ?

jandy123 · 2020-12-08T17:34:08Z

@m11tch

I noticed some config files in /home/cfg

I did see the stuff in /home/cfg. Tried to modify ota:0 in /home/cfg/capa.info in a hope to quiet the tuya app upgrade banner, but it didn't work....

jilleb · 2020-12-08T17:34:19Z

The connectiong was blocked all the time.

I started ppsapp from sd-card in the console with the --help commandline, hoping it would generate some useful output.But....... after closing it, the doorbell no longer boots 😆

jandy123 · 2020-12-08T17:36:03Z

Connection was blocked all the time here too.

@jilleb Hmm, no boot at all ???

guino · 2021-12-17T02:58:47Z

@McFlyPartages https://github.com/guino/BazzDoorbell/files/5672301/ppsapp-rtsp4.zip is only needed if your device will not have internet access (so you can use RTSP). If you block/disable internet access to the device you will need to create a local NTP server to set the time on the device (or the time on the video will be incorrect).

ONVIF is not available on your firmware version as far as I know.

McFlyPartages · 2021-12-18T10:07:25Z

Thanks for explain.

bkbartk · 2022-08-05T21:44:48Z

This is kind of an old post, But I'm going to ask anyway,

for the mqtt log parser to work, Am I modifying custom.sh correctly
I tried this

#!/bin/sh
if [ ! -e /tmp/customrun ]; then
 echo custom > /tmp/customrun
 cp /mnt/mmc01/passwd /etc/passwd
 /mnt/mmc01/busybox telnetd -l /bin/sh
 /mnt/mmc01/busybox httpd -c /mnt/mmc01/httpd.conf -h /mnt/mmc01 -p 8080
 if [ -e /mnt/mmc01/ppsapp ]; then
  PPSID=$(ps | grep -v grep | grep ppsapp | awk '{print $1}')
  kill $PPSID
  /mnt/mmc01/set record_enable 0
  /mnt/mmc01/set enable_event_record 1
  /mnt/mmc01/set onvif_enable 1
  /mnt/mmc01/ppsapp  2>&1 | /mnt/mmc01/log_parser.sh /mnt/mmc01/output.log &
 fi
 #/mnt/mmc01/offline.sh &
fi
if [ ! -e /tmp/cleanup`date +%Y%m%d` ]; then
 rm -rf /tmp/cleanup*
 touch /tmp/cleanup`date +%Y%m%d`
 /mnt/mmc01/cgi-bin/cleanup.cgi > /tmp/cleanup.log
fi

and this

#!/bin/sh
if [ ! -e /tmp/customrun ]; then
 echo custom > /tmp/customrun
 cp /mnt/mmc01/passwd /etc/passwd
 /mnt/mmc01/busybox telnetd -l /bin/sh
 /mnt/mmc01/busybox httpd -c /mnt/mmc01/httpd.conf -h /mnt/mmc01 -p 8080
 if [ -e /mnt/mmc01/ppsapp ]; then
  PPSID=$(ps | grep -v grep | grep ppsapp | awk '{print $1}')
  kill $PPSID
  /mnt/mmc01/set record_enable 0
  /mnt/mmc01/set enable_event_record 1
  /mnt/mmc01/set onvif_enable 1
  /mnt/mmc01/ppsapp &
 fi
 #/mnt/mmc01/offline.sh &
fi
if [ ! -e /tmp/cleanup`date +%Y%m%d` ]; then
 rm -rf /tmp/cleanup*
 touch /tmp/cleanup`date +%Y%m%d`
 /mnt/mmc01/cgi-bin/cleanup.cgi > /tmp/cleanup.log
fi
# debug info from ppsapp  forwarded to output.log
  /mnt/mmc01/ppsapp  2>&1 | /mnt/mmc01/log_parser.sh /mnt/mmc01/output.log &
# no debug info  
#  /mnt/mmc01/ppsapp 2>&1 | /mnt/mmc01/log_parser.sh /dev/null &

The fist one isn't producting any output at all, the 2nd one produces 1 really wierd line which I actually deleted, But Can regenerate if needed.
MQTT is not working in either case. I added username and passsword to the script.

guino · 2022-08-05T21:52:19Z

@bkbartk The 1st version should be what you need but you need to make sure you have ppsapp on the root of the SD card. So if you don't have a patched ppsapp on the root of the SD card just copy the original one from home/app/ppsapp to the root of the SD card and it should start it up correctly. You also want to make sure log_parser.sh is on the root of the SD card.

bkbartk · 2022-08-06T13:18:14Z

I didn't have ppsapp on the root of my SD card,
but if I do so, either patched or unpatchted it makes the complete devices unstable,

whenever I try to connect snap.cgi it times out or reboots,
when I pres the button in reboots instead of doing anything logical,
during telnet sometimes it reboots.

not sure why,
the output.log shows only one line
/m t/mmc01/ppsapp: ca 't load library 'libstdc++.so.6'
looks like I need to copy some other files as well, but I actually don't have a clue what is happening.

[edit]:
a small thing, but I noticed the devices crashes after ~65sec, regardless of my manual actions

guino · 2022-08-06T18:10:37Z

@bkbartk if your device is rebooting when you have a ppsapp file int he root of the SD card then either the ppsapp file is corrupted
OR your device may have the issue first found on firmware 2.9.0 (https://github.com/DanTLehman/orion_sc008ha).
You can try this to be sure:
1-Delete home directory and ppsapp from SD card
2-Make sure your custom.sh looks like this (regarding the ppsapp lines):

#!/bin/sh
if [ ! -e /tmp/customrun ]; then
 echo custom > /tmp/customrun
 cp /mnt/mmc01/passwd /etc/passwd
 /mnt/mmc01/busybox telnetd -l /bin/sh
 /mnt/mmc01/busybox httpd -c /mnt/mmc01/httpd.conf -h /mnt/mmc01 -p 8080
 if [ -e /mnt/mmc01/ppsapp ]; then
  PPSID=$(ps | grep -v grep | grep ppsapp | awk '{print $1}')
  kill $PPSID
  /mnt/mmc01/set record_enable 0
  /mnt/mmc01/set enable_event_record 1
  /mnt/mmc01/set onvif_enable 1
  /mnt/mmc01/ppsapp  2>&1 | /mnt/mmc01/log_parser.sh /mnt/mmc01/output.log &
 fi
 #/mnt/mmc01/offline.sh &
fi
if [ ! -e /tmp/cleanup`date +%Y%m%d` ]; then
 rm -rf /tmp/cleanup*
 touch /tmp/cleanup`date +%Y%m%d`
 /mnt/mmc01/cgi-bin/cleanup.cgi > /tmp/cleanup.log
fi

3-Boot the device, it should re-create the home directory
4-Copy home/app/ppsapp to the root of the SD card
5-Make sure you eject the SD card from the computer properly to prevent corruption of the ppsapp file after copying it (i.e. use eject button on system try)
6-Boot the device and see if the phone app, mjpeg,snap, RTSP work -- if not the issue may be similar to firmware 2.9.0

Let me know what you get.

PS: If you need to run a PATCHED ppsapp the only way would be modify the boot process to run the patched ppsapp first (Without having to kill it) using "Dan's method" mentioned on the link above (IT MUST BE MODIFIED FOR YOUR DEVICE -- don't try it as-is). If you only wish to get notifications (log_parser) I may have found a way to make this work without having to kill ppsapp (have to do more testing).

bkbartk · 2022-08-06T19:15:42Z

Sorry, The above didn't work, tried regenerating the home dir, that worked, but as soon as I copy PPSAPP to the root, my device crashes after 65 seconds(mesured by stopwatch, so not 100% acurate), I can use telnet, not RSTP, and if I'm quick I can create 1 snap before I get the Connection reset message.
To be sure the issue wan't musquito related I also tried the original custom.sh without logparser, but that doesn't make any difference.

I'm trying to work with to original file, not a patched one, because I think if the original doesn't work a patched one will fail for sure.

something I noticed is that my "env" file is completely different from what I changed last time, can that be the issue?
my file is now like this
bootargs=mem=37M console=ttyAMA0,115200n8 mtdparts=hi_sfc:192k(bld)ro,64k(env)ro,64k(enc)ro,64k(sysflg)ro,3136k(sys),4352k(app),320k(cfg) ppsAppParts=5 ppsWatchInitEnd - ip=\\${T//_/\\$\\'"\\\\x20"\\'}:::::";T=\\"sleep_5;mkdir_-p_/mnt/mmc01;mount_-t_vfat_/dev/mmcblk0p1_/mnt/mmc01;/mnt/mmc01/initrun.sh&\\";eval"
where originally(from stock) cmdline was like this
setenv bootargs mem=64M console=ttySAK0,115200n8 loglevel=10 mtdparts=spi0.0:256k(bld),64k(env),64k(enc),64k(sysflg),3m(sys),4032k(app),640k(cfg) ppsAppParts=5 ip=192.168.1.99:::255.255.255.0 eth=00:55:7b:b5:7d:f7 ethDisable usb_other
the first thing I notice is the memory difference
but fixing mem,console,loglevel and mtdparts doesn't do a thing.

Dan is talking about firmware 2.9.0, I have 5.2.2 on my device.

I have a patched version of my ppsapp generated with the ips file you provided me yesterday, maybe, that one I can use.

Still If I use Dan's method, at some point in custom.sh I tries to start another instance, which would be in issue I think?

guino · 2022-08-06T20:00:26Z

@bkbartk the different env file is only used to root the device - you can delete it after that (so current contents do not matter). You should however make sure the /proc/cmdline URL output is similar to your original one.

The changes Dan made for 2.9.0 actually only run ppsapp once because it runs ‘ppsapp-rtsp’and the custom.sh tries to find/run ‘ppsapp’. That process can probably be adjusted to your device but it is hard to know what would work without a copy of your device’s files (the home directory created on the SD card). For sure the env file would need to be adjusted with your cmdline settings.

bkbartk · 2022-08-06T20:31:31Z

/proc/cmdline is similar, so no issues there.

Here is a copy of my home directory, for as fas as I can see, there is no wifi info in it, and I left all passwords default.
home.zip

but maybe this method is an overkill, If I only have the log_parser. I probably could kill all internet connection using either hosts/pihole/routerblock.
I then only need to make sure the ntp server gets directed to my internal one. and I already found some line to modify that.
or am I missing something here?

I mean, decompiling the files is a lot of work for you and I don't know how much I can ask and my Linux knowledge is limited. So I might need some help with stuff which is really basic for a lot of people out here.

guino · 2022-08-07T03:18:35Z

@bkbartk I think you are right in that it is overkill to use a patched ppsapp for offline if you can handle it without patching. I believe I can make the log_parser work without the need for ppsapp in the root directory (by parsing the output of the ppsapp already running.
From what I could tell your startup scripts are mostly similar to the ones from 2.9.0 but there's no way of knowing if the existing files (for 2.9.0) would work unless you tried -- chances are they may need tweaking. Like we said I would only go that route if you think the non-patching offline method won't work for you (I would definitely recommend trying that).
Let me put together the method to get log parser working without re-running ppsapp and I'll let you know.

bkbartk · 2022-08-08T09:48:45Z

really thanks,
If there is anything I can do to help, provide additional information add, some testing, etc, please let me know.

guino · 2022-08-08T18:30:35Z

@bkbartk this seems to work:

1-Download this zip and extract it to your SD card's root directory: reredirect-logparser.zip
2-Adjust the log_parser with whatever parameters required for your notifications (motion/bell)
3-Make sure you don't have a ppsapp file in the root of the SD card
4-Adjust your custom.sh to be like this (new section is in the if [ ! -e /tmp/log ]; then block):

#!/bin/sh
if [ ! -e /tmp/customrun ]; then
 echo custom > /tmp/customrun
 cp /mnt/mmc01/passwd /etc/passwd
 /mnt/mmc01/busybox telnetd
 /mnt/mmc01/busybox httpd -c /mnt/mmc01/httpd.conf -h /mnt/mmc01 -p 8080
 if [ -e /mnt/mmc01/ppsapp ]; then
  PPSID=$(ps | grep -v grep | grep ppsapp | awk '{print $1}')
  kill $PPSID
  #/mnt/mmc01/set record_enable 0
  #/mnt/mmc01/set enable_event_record 1
  #/mnt/mmc01/set onvif_enable 1
  /mnt/mmc01/ppsapp &
 fi

 if [ ! -e /tmp/log ]; then
  PPSID=$(ps | grep -v grep | grep ppsapp | awk '{print $1}')
  /mnt/mmc01/busybox mkfifo /tmp/log
  /mnt/mmc01/reredirect -m /tmp/log $PPSID > /tmp/redir.log
  /mnt/mmc01/log_parser.sh /dev/null < /tmp/log &
 fi

 #/mnt/mmc01/offline.sh &
fi
if [ ! -e /tmp/cleanup`date +%Y%m%d` ]; then
 rm -rf /tmp/cleanup*
 touch /tmp/cleanup`date +%Y%m%d`
 /mnt/mmc01/cgi-bin/cleanup.cgi > /tmp/cleanup.log
fi

You can adjust custom.sh to your needs (i.e. run offline script, etc, the new block will launch the log_parser script using the output of the ppsapp already running.
I always suggest testing the mqtt commands on telnet to make srure they work then you can test it on the log parser.
If you want you CAN test the new commands (mkfifo, reredirect, log_parser.sh) in telnet before modifying custom.sh

bkbartk · 2022-08-08T20:29:01Z

Thnx,
Almost,
I'm able to dump the log in output.log, I also find the command for "button pressed", motion detection I still have to check.
The original app doesn't give any notifications anymore but I don't care about that.
I haven't tried to offline part yet.
But now I have an issue with mqtt
"mqtt_pub" doesn't accept credentials and just always returns that it has send data.
but nothing is retrieved because my mqtt server requires username and password.
"mosquitto_pub" from the zip above alsways returns "Illegal instruction (core dumped)"probaly because of an arm5 vs arm7 issue.
I had the same one with busybox.

guino · 2022-08-08T20:35:50Z

@bkbartk there's a version of mosquitto_pub (with credentials support) here: #35 (comment)

bkbartk · 2022-08-08T20:45:48Z

that one works

need to tweak a little but thanks a lot

bkbartk · 2022-08-09T18:43:40Z

I am completely offline now without patching.

So let me create a short summery so it might helps someone.
Before you go to the offline part, make sure MQTT works. At Least That's easier because while being offline your log gets flouded with data.
So check here for the mqtt to work
#4 (comment)
you can set output.log so you know what data gets grepped
/mnt/mmc01/log_parser.sh /mnt/mmc01/output.log < /tmp/log &
This helped me, also to know if log_parser was applied at all.

Then When MQTT works, you need to started blocking internet access.
I did this by creating a new group in my pi.hole and block all internet traffic Except for my mqtt server, ntp server and one required domain.

"Homeassistant" is both my MQTT as my ntp server.
"m2.tuyaeu.com"(domain may defer) is required at boot time, else RTSP won't work using ONVIF.
guino/Merkury1080P#9 (comment)
don't worry, We block it later.

If you don't have a pihole, you could block all addresses using the "hosts" file. but then you have to grep and list all and there are a lot of different tuya and amazon domains. I found them in pi hole log, but you may find them in tuya log, or your router log.

Then you need to modify "custom.sh" to enable to offline script

if [ ! -e /tmp/customrun ]; then
 echo custom > /tmp/customrun
 cp /mnt/mmc01/passwd /etc/passwd
 /mnt/mmc01/busybox telnetd -l /bin/sh
 /mnt/mmc01/busybox httpd -c /mnt/mmc01/httpd.conf -h /mnt/mmc01 -p 8080
 if [ -e /mnt/mmc01/ppsapp ]; then
  PPSID=$(ps | grep -v grep | grep ppsapp | awk '{print $1}')
  kill $PPSID
  /mnt/mmc01/set record_enable 0
  /mnt/mmc01/set enable_event_record 1
  /mnt/mmc01/set onvif_enable 1
  /mnt/mmc01/ppsapp  2>&1 | /mnt/mmc01/log_parser.sh /dev/null &
 fi
 if [ ! -e /tmp/log ]; then
  /mnt/mmc01/busybox ntpd -d -q -n -p homeassistant
  PPSID=$(ps | grep -v grep | grep ppsapp | awk '{print $1}')
  /mnt/mmc01/busybox mkfifo /tmp/log
  /mnt/mmc01/reredirect -m /tmp/log $PPSID > /tmp/redir.log
  /mnt/mmc01/log_parser.sh /dev/null < /tmp/log &
 fi
 /mnt/mmc01/offline.sh &
fi
if [ ! -e /tmp/cleanup`date +%Y%m%d` ]; then
 rm -rf /tmp/cleanup*
 touch /tmp/cleanup`date +%Y%m%d`
 /mnt/mmc01/cgi-bin/cleanup.cgi > /tmp/cleanup.log
fi

add the ntpd line
/mnt/mmc01/busybox ntpd -d -q -n -p homeassistant
make sure to uncomment the offline.sh line
/mnt/mmc01/offline.sh &
, and replace "/mnt/mmc01/output.log" with " /dev/null" once internet is down, there is so much data now, that you don't want this file anymore. If you want to read the log use
cat /tmp/log
and you can read realtime. but it's now really a bunch of ****
If you enable the offline part, notice that it will take about a minute longer to boot because of the restarting of the wifi.
but now "m2.tuyaeu.com" is available during boot, After boot, the hosts file is applied and the network is restarted.
after restarting the network "m2.tuyaeu.com" is also blocked.
So internet gets exposed for a really short while(20 seconds).

now I noticed that while being offline the button press is different so I need to modify the condition in the log_parser

#!/bin/sh

#DEBUG_FILE=/mnt/mmc01/output.log
DEBUG_FILE=$1

# contains(string, substring)
#
# Returns 0 if the specified string contains the specified substring,
# otherwise returns 1.
contains() {
    string="$1"
    substring="$2"
    if test "${string#*$substring}" != "$string"
    then
        return 0    # $substring is in $string
    else
        return 1    # $substring is not in $string
    fi
}

main() {
    IFS='$\n'
    echo -n "" > $DEBUG_FILE
    while true; do
        read -r BUF;
        if [ $? -ne 0 ]; then
            sleep 1;
            continue
        fi
        if contains "$BUF" "frames exceed max number"; then
            echo $(date +"%Y-%d-%m %H:%M:%S") + " motion detected" >> /mnt/mmc01/mqttsent.log
            /mnt/mmc01/mosquitto_pub -h homeassistant -u MyUser -P MyPassword -m "ON" -t doorbell/motion
        elif contains "$BUF" "doorbell pressed" || contains "$BUF" "external_charm_ctrl = 1"; then
            echo $(date +"%Y-%d-%m %H:%M:%S") + " doorbell push button" >> /mnt/mmc01/mqttsent.log
            /mnt/mmc01/mosquitto_pub -h homeassistant -u MyUser -P MyPassword -m "ON" -t doorbell/button
        #else
            #echo "Unknown cmd: $BUF"
        fi
        echo $BUF >> $DEBUG_FILE
    done
}

main

If the doorbell is online I need to grep "doorbell pressed" but when it's offline I need to grep "external_charm_ctrl = 1"
Check the code.

now you can use your device offline without patching.

I hope my explanation is clear enough and it helps someone.

jonesMeUp · 2022-11-24T01:49:58Z

I have started to write an universal initrun.sh that handles all needed stuff for patched offline devices.
No more edit in different files for your custom settings, or have different shell files - all is done in initrun.sh.
Would be very nice if you can test it on your device and perhaps add your working env file to the thread when its not supported right now. Any questions can be asked there...

cweiland · 2024-10-14T20:30:13Z

Hello everyone,
I'm pretty sure I ve patched my ppsapp.
But it's still running at boot and I dont really understand why
init-+-3*[busybox]
|-2*[cmd_router]
|-dropbearmulti---dropbearmulti---sh---pstree
|-initrun.sh-+-initrun.sh---sleep
| |-ppsapp-+-{charm_loop}
| | |-{cmmod}
| | |-{doorbell_push}
| | |-{gpiolistener}
| | |-{led-service}
| | |-{lpc_task}
| | |-{onvif_webserve}
| | |-2*[{ppsapp}]
| | |-{sdcard_loop}
| | |-{sleeping_monit}
| | |-{sys_timer}
| | |-{watchdog}
| | |-{wifi_patch}
| | |-{wk_th-0}
| | |-{wk_th-1}
| | -{wpa_cli_task} | -sleep
|-login
|-ppsapp-+-{aencTask}
| |-{aiTask}
| |-{ai_capture}
| |-{charm_loop}
| |-{cmmod}
| |-{dnTask}
| |-{doorbell_push}
| |-{gpiolistener}
| |-{init_8090_port}
| |-{led-service}
| |-{lpc_task}
| |-{mdCall}
| |-{mdTask}
| |-{onvif_webserve}
| |-{osdTask}
| |-2*[{ppsapp}]
| |-{sdcard_loop}
| |-{sleeping_monit}
| |-{sys_timer}
| |-{talkTask}
| |-{vencTask}
| |-{venc_capture}
| |-{venc_encode}
| |-{vi_change_fps}
| |-{watchdog}
| |-{wifi_patch}
| |-{wk_th-0}
| |-{wk_th-1}
| -{wpa_cli_task} |-ppsdsry |-udevd---udevd |-udhcpc -wpa_supplicant
Can you help me ?

cweiland · 2024-10-14T20:32:30Z

For info :
hardware_version: BE8S_A2_V10_433
software_version: 4.0.7
firmware_version: ppstrong-a3-tuya2_general-4.0.7.20210513
model: Bell 8S

cweiland · 2024-10-16T15:47:54Z

Another try another message :
`[00:09:22.321 INFO pps_device_factory.c:64]we are now in factory mode
[01-01 18:21:34:327 TUYA Debug][wf_nw_cfg.c:100] Set New Channel 1
[01-01 18:21:34:357 TUYA Err][iot_httpc.c:2123] http url and https url are all NULL
[01-01 18:21:34:358 TUYA Debug][iot_httpc.c:909] Post Data: {"type":4,"t":559}
[01-01 18:21:34:358 TUYA Debug][iot_httpc.c:920] Post URL: ?a=tuya.device.ipc.p2p.config.get&devId=&et=1&t=559&v=2.0&sign=b9495009f3abbd72ce4614df7057f1f4
[01-01 18:21:34:359 TUYA Err][httpc.c:485] tcp connect failed ?a=tuya.device.ipc.p2p.config.get&devId=&et=1&t=559&v=2.0&sign=b9495009f3abbd72ce4614df7057f1f4:80 errno=101
[01-01 18:21:34:359 TUYA Err][http_inf.c:47] Open session failed: ?a=tuya.device.ipc.p2p.config.get&devId=&et=1&t=559&v=2.0&sign=b9495009f3abbd72ce4614df7057f1f4 (-7)
[01-01 18:21:34:359 TUYA Err][iot_httpc.c:926] Post Data Return Fail.-706
[01-01 18:21:34:359 TUYA Err][tuya_ipc_p2p_common.c:282] get p2p auth failed
[01-01 18:21:34:359 TUYA Err][tuya_ipc_p2p_common.c:371] get p2p auth retry cnt[495]
[01-01 18:21:34:407 TUYA Err][tuya_ipc_streamer.c:1228] wait gw init ...
[01-01 18:21:34:617 TUYA Err][tuya_ipc_streamer.c:1228] wait gw init ...
[00:09:22.721 INFO pps_device_factory.c:64]we are now in factory mode

`

jonesMeUp · 2024-10-18T15:21:09Z

hi cweiland,
my doorbell doesn't exist anymore because it was destroyed by a burglar while "cleaning" my home.
so i can't do any more testings on the device.

but on a short look on your posting: your ppsapp doesn't seem to be patched, because i see TUYA errors.
so you have to start at the beginning again.

cweiland · 2024-10-20T14:56:19Z

Find a way to disable registration
`int tuya_ipc_p2p_auth_proc(void)

{
int returnValueAuth;
char *__s;
size_t sVar1;

returnValueAuth = FUN_00104a04(4);
__s = (char *)FUN_000e47c4(0);
if (_s == (char *)0x0) {
FUN_000d4b00(0,0,
"/var/lib/jenkins/workspace/Release_IPC_SDK/src/tuya_ipc_sdk/p2p/common/tuya_ipc_p2 p_common.c"
,0x11a,"tuya_ipc_get_p2p_auth_proc","get p2p auth failed");
FUN_000e3d1c(0);
returnValueAuth = 0;
}
else {
if (returnValueAuth == 0) {
FUN_000d4b00(0,4,
"/var/lib/jenkins/workspace/Release_IPC_SDK/src/tuya_ipc_sdk/p2p/common/tuya_ipc p2p_common.c"
,0x121,"tuya_ipc_get_p2p_auth_proc","SY P2P AUTH:%s",__s);
sVar1 = strlen(__s);
thunk_FUN_001613a4("p2p_auth_info",__s,sVar1 + 1);
Free(__s);
}
FUN_000e3d1c(0);
}
return returnValueAuth;
}`

The idea is to force returnValueAuth to 0
With this rtsp server starts !!

jilleb mentioned this issue Dec 8, 2020

No programmer, No UART, No problem! #2

Closed

guino mentioned this issue Jan 3, 2022

Doorbell ntp timezone is set #67

Open

sanscorp1 mentioned this issue Feb 14, 2022

Offline patch request guino/ppsapp-rtsp#35

Closed

guino mentioned this issue Aug 11, 2022

4.0.7: solution(s) for my device #82

Open

guino mentioned this issue Aug 24, 2022

Damn, I bricked my camera! guino/Merkury1080P#36

Open

guino mentioned this issue Oct 29, 2022

Rooted version FW 5.0.5 by hw programmer - sdcard option in progress ( was finding correct firmware version (Bricked device) ) #62

Open

jonesMeUp mentioned this issue Nov 9, 2022

OFFLINE: common code base #90

Open

desiue mentioned this issue Nov 19, 2022

ppsapp patch request guino/Merkury1080P#45

Open

tosiara mentioned this issue Dec 20, 2022

Is it possible to disable ssl/tls validation? guino/ppsapp-rtsp#47

Closed

Off-cloud research #4

Off-cloud research #4

Comments

jilleb commented Dec 8, 2020

m11tch commented Dec 8, 2020 • edited Loading

jilleb commented Dec 8, 2020 • edited Loading

jilleb commented Dec 8, 2020

guino commented Dec 8, 2020

guino commented Dec 8, 2020

m11tch commented Dec 8, 2020 • edited Loading

jandy123 commented Dec 8, 2020

jandy123 commented Dec 8, 2020 • edited Loading

jilleb commented Dec 8, 2020

guino commented Dec 8, 2020

m11tch commented Dec 8, 2020 • edited Loading

jandy123 commented Dec 8, 2020

jandy123 commented Dec 8, 2020

jandy123 commented Dec 8, 2020 • edited Loading

guino commented Dec 8, 2020

m11tch commented Dec 8, 2020 • edited Loading

jandy123 commented Dec 8, 2020 • edited Loading

jandy123 commented Dec 8, 2020

jandy123 commented Dec 8, 2020

m11tch commented Dec 8, 2020

jandy123 commented Dec 8, 2020 • edited Loading

jilleb commented Dec 8, 2020 • edited Loading

jandy123 commented Dec 8, 2020

m11tch commented Dec 8, 2020

jandy123 commented Dec 8, 2020 • edited Loading

m11tch commented Dec 8, 2020 • edited Loading

jandy123 commented Dec 8, 2020

jilleb commented Dec 8, 2020

jandy123 commented Dec 8, 2020

guino commented Dec 17, 2021

McFlyPartages commented Dec 18, 2021

bkbartk commented Aug 5, 2022

guino commented Aug 5, 2022

bkbartk commented Aug 6, 2022 • edited Loading

guino commented Aug 6, 2022 • edited Loading

bkbartk commented Aug 6, 2022

guino commented Aug 6, 2022

bkbartk commented Aug 6, 2022

guino commented Aug 7, 2022 • edited Loading

bkbartk commented Aug 8, 2022

guino commented Aug 8, 2022 • edited Loading

bkbartk commented Aug 8, 2022

guino commented Aug 8, 2022

bkbartk commented Aug 8, 2022

bkbartk commented Aug 9, 2022

jonesMeUp commented Nov 24, 2022

cweiland commented Oct 14, 2024

cweiland commented Oct 14, 2024

cweiland commented Oct 16, 2024

jonesMeUp commented Oct 18, 2024

cweiland commented Oct 20, 2024

m11tch commented Dec 8, 2020 •

edited

Loading

jilleb commented Dec 8, 2020 •

edited

Loading

m11tch commented Dec 8, 2020 •

edited

Loading

jandy123 commented Dec 8, 2020 •

edited

Loading

m11tch commented Dec 8, 2020 •

edited

Loading

jandy123 commented Dec 8, 2020 •

edited

Loading

m11tch commented Dec 8, 2020 •

edited

Loading

jandy123 commented Dec 8, 2020 •

edited

Loading

jandy123 commented Dec 8, 2020 •

edited

Loading

jilleb commented Dec 8, 2020 •

edited

Loading

jandy123 commented Dec 8, 2020 •

edited

Loading

m11tch commented Dec 8, 2020 •

edited

Loading

bkbartk commented Aug 6, 2022 •

edited

Loading

guino commented Aug 6, 2022 •

edited

Loading

guino commented Aug 7, 2022 •

edited

Loading

guino commented Aug 8, 2022 •

edited

Loading