Import *.rrd from https://github.com/wiedehopf/graphs1090#graphs1090

[dranreb38]

I would like to keep all my data collected by the script of https://github.com/wiedehopf/graphs1090#graphs1090 and then import it into your project. I tried to copy the rdd folder into your installation (/var/lib/collectd/rrd/localhost) but without success! Do you have any idea how to import *.rrd files

[jprochazka]

Graphs1090 would in fact appear to use a different data format than this project as outlined here?

However, being that @wiedehopf's graphs1090 is a much more up to date version of dump1090-tools/ADS-B Portal graphs. I would recommend sticking with his graphs for the time being. I will go ahead and create a ticket geared towards adding graphs1090 support to the install scripts. Adding an option to install graphs1090 instead of using the existing graphs system should be a trivial task and one I am willing to undertake over continued support for a system I am looking to replace in v3.x*.

I am currently testing the next release which will include VDL Mode 2 support and should be able to squeeze in support for graph1090 for this release as well. How does a release adding said support sometime mid to late this week sound?

* As mentioned in previous announcements one of my goals with v3.x is to replace the dated Collectd/RRDTools graphs currently implemented within this project with a more modern/versatile solution. When implemented I will be sure to add proper data conversion options for both the current ADS-B Portal rrd files as well as those generated by graphs1090.

[wiedehopf]

graphs1090 is still collectd / rrd.
the difference in DB format is mostly in relation to things like range limited to 590 / message rate limit values people were hitting in weird usecases.
So most those limits are gone. That required such a DB format reset.
Similar for when people asked for longer time graphs, there is now graphs for up to 10 yrs.

There is no good replacement for rrd, it all hogs lots of storage / doesn't seem as robust.

Not all the data will be there, but you can change from the mutability graphs install to graphs1090 and it'll carry over most of the existing data.
And it'll work without resetting the format, but it will be limited to 2 yrs and some other limitations.

[jprochazka]

Just an update, I started working on offering the ability to install Readsb as a decoder option so that a graphs1090 installation option would be fully functional as far as displaying data is concerned which wiedenhopf pointed out would be an issue. I ran into a problem however in that Readsb does not play nice with Ubuntu with the recommendation to use Debian or Docker images as the solution. Both options go against the goals of this project so for the time being I am going to have to shelve this solution.

I tried to copy the rdd folder into your installation (/var/lib/collectd/rrd/localhost) but without success!

the difference in DB format is mostly in relation to things like range limited to 590 / message rate limit values people were hitting in weird usecases.

If in fact the only changes to the rrd databases are size constraints...

@dranreb38 what does the file structure look like? It should look something like this.

/var/lib/collectd/rrd/localhost/aggregation-cpu-average
/var/lib/collectd/rrd/localhost/df-root
/var/lib/collectd/rrd/localhost/disk-mmcblk0
/var/lib/collectd/rrd/localhost/disk-mmcblk0
/var/lib/collectd/rrd/localhost/interface-eth0
/var/lib/collectd/rrd/localhost/memory
/var/lib/collectd/rrd/localhost/thermal-thermal_zone0
/var/lib/collectd/rrd/localhost/thermal-thermal_zone0
/var/lib/collectd/rrd/localhost/interface-wlan0
/var/lib/collectd/rrd/localhost/dump1090-localhost
/var/lib/collectd/rrd/localhost/dump1090-localhost
/var/lib/collectd/rrd/localhost/dump1090-localhost
/var/lib/collectd/rrd/localhost/dump1090-localhost
/var/lib/collectd/rrd/localhost/dump1090-localhost
/var/lib/collectd/rrd/localhost/dump1090-localhost

There is no good replacement for rrd, it all hogs lots of storage / doesn't seem as robust.

I am leaning towards replacing rrdtools with Prometheus in the new portal. I am planning on moving graph generation from the backend/receiver to the frontend/client side so the rrdtools package will lose what little value add it brought to the table as far as the portal will be concerned. Tools such as Grafana can connect to Prometheus as a data source fulfilling a long-standing request for a customizable graphing solution.

[wiedehopf]

graphs1090 works fine with dump1090-fa

If you do an automated build for Ubuntu 24 .. just do

sed -i -e 's/librtlsdr0/librtlsdr2/g' debian/control

before building.
problem solved.

[wiedehopf]

This should no longer be needed due to a PR to the control file, apparently you can simply use | to use one package or another.

[jprochazka]

Did not know that but is good to know for occasions such as this.

[wiedehopf]

I don't have Ubuntu24 on hand but maybe you could test?

wiedehopf/readsb#77

It's already in dev, there is also the stale branch which i use for https://github.com/wiedehopf/adsb-scripts/wiki/Automatic-installation-for-readsb

But either branch should be fine, it's very rare for problematic versions to be pushed to dev :)

[jprochazka]

I sure can. Actually, about to test it out now. Will report back shortly.

[jprochazka]

I am away from home at the moment so I do not have access to an RTL-SDR dongle at the moment but I can confirm that Readsb is in fact building on both Ubuntu Noble Numbat and Debian Bookworm. There is one thing missing from your build instructions and that is at sudo apt install --no-install-recommends --no-install-suggests -y ... in the README.md file you might want to add the fakeroot package.

Note: I did a manual build using the install script located in the vdlm2 branch of this project.

[dranreb38]

@jprochazka , yes

[dranreb38]

Do I need to run a command to integrate the *.rrd files? I also did an export in xml

[jprochazka]

Everything looks proper as far as paths are concerned.
Do you get any errors when running the graph generation command we could use to further investigate things?

bash ~/adab-receiver/build/graphs/make-collectd-graphs.sh 1h
 
If you exported the data as XML you will need to use the rrdrestore command in order to create proper rrd databases.
https://oss.oetiker.ch/rrdtool/doc/rrdrestore.en.html

[dranreb38]

@jprochazka
I transferred to your project all the *.rrd files from my "old RPI" in the folder /var/lib/collectd/rrd/localhost/dump1090-localhost in the new one. Then launched the script "make-collectd-graphs.sh 1h" but I don't have the data from my old RPI yet, maybe I have to wait a little? ......
there are errors but it's normal because I work with an SSD and not an SD card.

For 30 Days make-collectd-graphs.sh 30d ???

[jprochazka]

Adding an option to install graphs1090 instead of using the existing graphs system should be a trivial task

I merged the vdlm2 branch into the master branch tonight which adds an option to install Graphs1090 in case you would rather stick to that. I did not modify the PHP portal so you will want to disable the default graphs link and create a custom link to the Graphs1090 page in the PHP portal administration area.

[jprochazka]

If you are in fact still having issues and do not mind sharing your rrd data with me, I would be willing to take a shot at it if you can get me a copy of said rrd data. Let me know if you would like to go that route so we can arrange a way to share said data if you are not comfortable sharing it here.

[dranreb38]

@jprochazka
Send url for download data my history
Regards

[jprochazka]

Sorry for the delay getting back to you. I have been settling into a new job this week. I wish they had a DM feature oh well I will leave this link for you to use. Hopefully it won't get abused.

https://www.dropbox.com/request/drTxGqi5QcV5XP4Iqxg5

[dranreb38]

Hello Joseph, thanks for the link I upload my data files in *.rrd format from my feeder (dranreb38 / T-LFH7) Thank you for your work!
Kind regards

[dranreb38]

Hello Joseph,
have you found a way to integrate the history of the graphs ?
Kind regards

[jprochazka]

I am so sorry. I saw that the question was answered and thought you resolved the issue. I will begin looking into it today. I went ahead and unmarked the question as answered until we get the issue resolved.