Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Spinnaker SDK V1.20 is no longer available from FLIR web site. #3

Closed
LeeYangLBLBCS opened this issue Jul 19, 2020 · 47 comments
Closed

Spinnaker SDK V1.20 is no longer available from FLIR web site. #3

LeeYangLBLBCS opened this issue Jul 19, 2020 · 47 comments

Comments

@LeeYangLBLBCS
Copy link

The only versions available is V2.0 and V1.29 from FLIR:
https://meta.box.lenovo.com/v/link/view/a1995795ffba47dbbe45771477319cc3
Is there any plan to update the drive to work with the newer version SDK?
Would you happen to have an older version archived?
@LeeYangLBLBCS
Copy link
Author

Spinnaker SDK v1.20 can still be obtained from FLIR if requested at this time.

@MarkRivers
Copy link
Member

Sorry for the delay in replying.

I downloaded Spinnaker 2.0 from FLIR yesterday for both Windows and Linux. I will use that version in ADSpinnaker. Hopefully I can do it tomorrow.

@LeeYangLBLBCS
Copy link
Author

LeeYangLBLBCS commented Jul 22, 2020 via email

@MarkRivers
Copy link
Member

However it doesn't work with my camera. FLIR oryx-10G-51S5.
I think it because the firmware is too new to work with the old Spinnaker.

I have that camera and it is working fine with v1.20 and ADSpinnaker.

There is a screen shot of the medm screen running that camera in ADSpinnaker, which shows the model, and the SDK version here: https://areadetector.github.io/master/ADSpinnaker/ADSpinnaker.html#medm-screens

If you are having problems please post the exact error message.

@LeeYangLBLBCS
Copy link
Author

LeeYangLBLBCS commented Jul 22, 2020 via email

@MarkRivers
Copy link
Member

Is the firmware on your camera 1710.0.0.0?

Yes, you can see that in the medm screen referenced in my last message. What is the firmware version of your camera?

@LeeYangLBLBCS
Copy link
Author

LeeYangLBLBCS commented Jul 26, 2020 via email

@MarkRivers
Copy link
Member

Those errors are coming because the camera is defaulting to jumbo packets. Go to the Features #1 screen and change the PacketSize (which is probably about 8000) to 1500.

This is a common problem with ADSpinnaker. I will add the packet size to the standard parameters, and add it to autosave.

@MarkRivers
Copy link
Member

Your attachments are not showing up. You cannot add them from e-mail, you need to add them directly on Github.

You cannot see the images in medm. I recommend using the ImageJ plugin from ADViewers.

@LeeYangLBLBCS
Copy link
Author

I assume you meant the "Camera-specific features" drop down list "Featuers #1"?
I tried it, and Features 2,3,4,... but got errors:

Cannot open related display:
$(C)-features_1.adl
Check EPICS_DISPLAY_PATH

I suspect the parameter $(C) is not instantiated properly, though I don't know where it's being set.

Also, do these Features require the "aravisGigE" module to be compiled? (It didn't).

image
medm error specific features

@MarkRivers
Copy link
Member

I assume you meant the "Camera-specific features" drop down list "Featuers #1"?
I tried it, and Features 2,3,4,... but got errors:

Cannot open related display:
$(C)-features_1.adl
Check EPICS_DISPLAY_PATH

I suspect the parameter $(C) is not instantiated properly, though I don't know where it's being set.

You need to pass the C macro when you load the ADSpinnaker screen, the same way you pass P and R. The C macro should be the string that is specific to your camera, in this case you would pass:

C=FLIR_ORX_10G_51S5M

You can look at ADTop.adl for examples.

Also, do these Features require the "aravisGigE" module to be compiled? (It didn't).

No, you never need aravisGigE. You do need to build the Linux package aravis if you need to add support for a new camera that is not already in ADGenICam, because you need to read the XML file from the camera and build the EPICS database and medm screens. But for your Oryx camera those files are already in ADGenICam.

You can use either ADSpinnaker or ADAravis to control that camera.

@MarkRivers
Copy link
Member

When I started "Spinnaker IOC", I keep getting errors with image acquisition:
2020/07/25 20:14:54.695 ADSpinnaker::grabImage error GetImageStatus returned 9

My previous answer to this question, where I said to change the PacketSize is incorrect. I forgot you are using a USB camera, not a GigE camera.

For USB cameras there are Linux system settings that need to be changed in order to stream images larger than 2 MB. Those are described here: https://areadetector.github.io/master/ADGenICam/ADGenICam.html#linux-usb-and-gige-system-settings

Have you changed the settings described on that page?

@LeeYangLBLBCS
Copy link
Author

LeeYangLBLBCS commented Jul 26, 2020 via email

@LeeYangLBLBCS
Copy link
Author

LeeYangLBLBCS commented Jul 26, 2020 via email

@MarkRivers
Copy link
Member

MarkRivers commented Jul 26, 2020 •
edited
Loading

I'm using GigE, not USB.

Sorry, that's right the Oryx is 10 GBit Ethernet, not USB.

The error is not happening nearly as often, but still occasionally, about 1/min, as compared to 1/few seconds.

What kind of system is this, i.e. what CPU type and how many cores? Send the output of

more /proc/cpuinfo

I am able to stream with that camera with very few if any dropped frames on a system with 16 cores and Intel(R) Xeon(R) W-2145 CPU @ 3.70GHz.

Here is a screen shot of my camera. I have set it for 163 frames/s, ImageMode=Multiple and NumImages=20000.

image

I collected all 20000 frames without seeing the message you reported.

Is your camera on a dedicated NIC? Mine is, and that is a good idea. It does not share Ethernet bandwidth with any other devices.

@LeeYangLBLBCS
Copy link
Author

LeeYangLBLBCS commented Jul 26, 2020 via email

@MarkRivers
Copy link
Member

I couldn't change the exposure time and frame rate for some reason. So I can't do the same test you did.
When I tried to change them, the value reverts back quickly.
I can change some of the other parameters though, such as trigger mode, trigger input etc.
Maybe there are file permission problems in my setup?

It is not a file permissions problem, it is because you have set ExposureAuto=Continuous. So it is continually adjusting the exposure time. You should set ExposureAuto=Off. Then you should be able to use the same settings I used.

Your computer does not look bad, 8 cores at 3.7 GHz.

@LeeYangLBLBCS
Copy link
Author

that's what I thought at first. But no that's not it.
I made sure the exposure is off.
What's more interesting is that I can change exposure time from command line:
lyang@lyangUbuntu:~/epics/flir_setup$ caput 13SP1:cam1:GC_ExposureTime 0.03
Old : 13SP1:cam1:GC_ExposureTime 0.02
New : 13SP1:cam1:GC_ExposureTime 0.03
And the exposure time is updated correctly in "features #1" window as I watched it changing.
But I can't change it from either the main MEDM window or the feature #1.
image
image

@LeeYangLBLBCS
Copy link
Author

Maybe I am confused which is exposure time.
There are two numbers next to "ExposureTime" label.
The left number can't be changed - may be that's the measured value?
The right number changes when I change it from command line through caget.
caput 13SP1:cam1:GC_ExposureTime
I thought there is a way to change it from the GUI, No?

@MarkRivers
Copy link
Member

MarkRivers commented Jul 26, 2020 •
edited
Loading

The problem is that you have FrameRateEnable=Yes, and you have set the FrameRate=71.847. That means that your requested ExposureTime=0.015 is impossible, because at that FrameRate the longest possible ExposureTime is 1/71.847=0.0139 seconds. You can set FrameRateEnable=No, and then the frame rate will be controlled by the requested ExposureTime, up to the maximum possible FrameRate.

Note that you have set PixelFormat=Mono16. That is slower, so you can't get 163 frames/s in that mode. Change to Mono8 and you should be able to get 163 frames/s.

The left number can't be changed - may be that's the measured value?
The right number changes when I change it from command line through caget.
caput 13SP1:cam1:GC_ExposureTime

There are 2 PVs that control the ExposureTime. The first is the areaDetector abstraction called AcquireTime. That is on the main ADSpinnaker screen and that is what you should be changing. GC_ExposureTime is the low-level GenICam parameter. When you change AcquireTime it actually writes to the same GenICam feature as GC_ExposureTime, but it does additional things, so you should use AcquireTime. If you use the middle mouse button in medm it will show you the name of the PV. You can also use the right button and then click PV Info with the left button on a widget you want to see.

@LeeYangLBLBCS
Copy link
Author

thanks. Mouse click info really helps.
Though the frame rate enable=Off didn't do anything for me.
image

@MarkRivers
Copy link
Member

Do you know why I can't change acquisition time?
I think it's related to the initial error message I got when ioc starts:
2020/07/26 10:04:24.763 Param[ACQ_TIME] GenICamFeature::write: node ExposureTime is not writable
2020/07/26 10:04:24.800 Param[ACQ_PERIOD] GenICamFeature::write: node AcquisitionFrameRate is not writable

Those errors were because you had ExposureTimeAuto=Yes.

@MarkRivers
Copy link
Member

Your fonts are messed up now and hard to read. Close the window and re-open it and the fonts should go back to normal.

Put it in ImageMode=Continuous and start Acquire. You should then be able to change the exposure time, and the Image Rate it reports should change. Try 0.1, 0.01, 0.5, etc.

Are you pressing Enter after you change the AcquireTime? You need to do that without moving the mouse out of that widget before pressing Enter.

@LeeYangLBLBCS
Copy link
Author

LeeYangLBLBCS commented Jul 26, 2020 via email

@MarkRivers
Copy link
Member

Yes, medm has a different interface that way. The idea is that if the user moves the mouse out of the widget without pressing Enter the value reverts to its previous version. That is to prevent mistakes from the operators in the control room, they need to press Enter first.

Can you configure things like I did, and get an actual frame rate of 163 frames/s? Post another screen shot.

@LeeYangLBLBCS
Copy link
Author

thanks. Mouse click info really helps.
Though the frame rate enable=Off didn't do anything for me.
image

@LeeYangLBLBCS
Copy link
Author

I did the same test using the same params from yours.
There is one error occurring once at around 13-kth frame.

2020/07/26 12:38:40.844 ADSpinnaker::grabImage error GetImageStatus returned 9

It must be caused by non ideal condition of my computer hardware, I guess.

image

@MarkRivers
Copy link
Member

MarkRivers commented Jul 26, 2020 •
edited
Loading

Notice that although you asked for 163 frames/s you are only getting 144.554 frames/s. That is because you are in 10-bit mode, not 8-bit mode. You need to open Features#2 screen and on the lower-left change AdcBitDepth from 10 to 8. Then you should be able to get the full 163 frames/s.

What application will you be using this camera for? Is it for the 8.3.2 micro-tomography beamline?

@LeeYangLBLBCS
Copy link
Author

I did the same test using the same params from yours.
There is one error occurring once at around 13-kth frame.

2020/07/26 12:38:40.844 ADSpinnaker::grabImage error GetImageStatus returned 9

It must be caused by non ideal condition of my computer hardware, I guess.

image

@LeeYangLBLBCS
Copy link
Author

I changed to 8 bit ADC and got the same frame rate and without any error this time.
And yes this development is intended for use at 8.3.2 uT, though we are not sure if the camera is too noisy until we take real images.
We've been using a slower camera (PCO), but less noisy.
Also we probably have to use 10 bit or even 12 bit, and mono16 in real experiments.
So we won't be able to achieve the same frame rate.
image

@MarkRivers
Copy link
Member

MarkRivers commented Jul 26, 2020 •
edited
Loading

If you are going to do tomography I would recommend these settings.

image

On the Features#2 screen I have selected AdcBitDepth=10. Note that on this screen I have selected PixelFormat=Mono12Packed. That sends a 2 12-bit pixels in 3 bytes, which is faster than sending 16-bit pixels in 4 bytes. I also selected ConvertFormat=Mono16, so it convert the 12-bit pixels to 16-bit pixels on the IOC computer, not in the camera. In those mode the maximum frame rate is almost 110 frames/s. I can't quite get that, but I do get 105 frames/s.

And yes this development is intended for use at 8.3.2 uT, though we are not sure if the camera is too noisy until we take real images.
We've been using a slower camera (PCO), but less noisy.**

I am not sure you will actually see any difference in the noise of the Oryx and the PCO. The read-noise of the PCO is lower, but in most tomography experiments that makes absolutely no difference. The reason is that most tomography images are limited by the shot noise, not by the read noise. The full-well capacity of the Oryx is 10,435 e-, and the read noise is 2.31 e-. If you adjust the exposure time so that the air is almost saturating the camera (10,000 e-) then the shot noise in the air pixels is sqrt(10000)=100. So the noise in each pixel is 100 e-, or 1% of the signal. This is 43 times larger than the read noise, so the read noise it irrelevant in this case. If the most absorbing part of the sample absorbs 95% of the beam then the signal in that pixel will be 500 e-, and the noise in those pixels will be sqrt(500)=22.3 e-. This is still almost 10X more than the read noise. Only if the signal in a pixel is less than 10e- will the read noise be comparable to the shot noise.

In terms of dynamic range, which controls how many ADC bits you need, if we use the example of air=10000 e- and darkest pixel=500, then we need to capture the noise in the darkest pixels (22e-) and intensity of the brightest pixels (10000 e-). That is a dynamic range of 10000/22 = 455, which is just less than 9 bits. So a 10-bit ADC is plenty to capture the full dynamic range of those images.

@LeeYangLBLBCS
Copy link
Author

LeeYangLBLBCS commented Jul 26, 2020 via email

@MarkRivers
Copy link
Member

You save the images with the areaDetector file plugins. On the medm screen go to Plugins/File. There are plugins to save files in HDF5, Nexus, netCDF, TIFF, JPEG, and other formats. For tomography you will want HDF5. You can configure the HDF5 file layout via an XML file. https://areadetector.github.io/master/ADCore/NDPluginFile.html. You can configure additional metadata to save with each file, or each image in the file, with another XML file.

@LeeYangLBLBCS
Copy link
Author

I can save image to tiff file pluggin without much trouble.
But when I tried HDF5, the contents is empty.
here is my H5 screen shot:
image
Can you see anything obviously wrong?
Here is the content of a H5 file in HDF5View:
image

@MarkRivers
Copy link
Member

That looks OK.

Please list the contents of that directory with “ls -l“.
Also run the command “h5dump —header _abcdesss_22.h5” and show that output.

@LeeYangLBLBCS
Copy link
Author

the file size is about 5MB.
the output of h5dump is in attached file.

lyang@lyangUbuntu:~/images$ ls -l
total 4940
-rw-r--r-- 1 lyang lyang 5043608 Jul 26 19:20 _abcdesss_22.h5

h5dump.txt

@MarkRivers
Copy link
Member

ls -l and h5dump indicate that the HDF5 file is the right size, and contains the expected data in /entry/data/data. I think there is something wrong either with the HDF5Viewer itself, or with the way you are using it.

Try this command:

h5dump —dataset /entry/data/data _abcdesss_22.h5

That should dump the image data itself. No need to post all of it, just make sure the data looks reasonable.

@MarkRivers
Copy link
Member

I just saved an HDF5 file using the following settings:

image

It displays fine with HDFView 2,14

image

It looks like the interface has changed a bit bettween 2.11 (yours) and 2.14 (mine). Perhaps you need to double-click on the filename or do something else to expand the view to show what is in the file?

@LeeYangLBLBCS
Copy link
Author

LeeYangLBLBCS commented Jul 27, 2020 via email

@LeeYangLBLBCS
Copy link
Author

LeeYangLBLBCS commented Aug 7, 2020 via email

@MarkRivers
Copy link
Member

I am not sure the SpinView errors and the EPICS errors are related. The SpinView errors look like they are from QT, i.e. GUI related. The EPICS error could be data transmission related. Is this Linux or Windows? If Linux did you set those system parameters for Ethernet buffers on the new system?

@LeeYangLBLBCS
Copy link
Author

It's linux, ubuntu 18.04Added /etc/sysctl.conf with these two lines:net.core.rmem_max=8388608
net.core.rmem_default=8388608

FLIR CAMERA model: ORX-10G-51S5M-C
image

@MarkRivers
Copy link
Member

Did you reboot Linux after changing those settings? Did you read the current run-time values to make sure those values are what is currently in use?

@MarkRivers
Copy link
Member

Is the camera using jumbo packets? That is in one of the camera specific screens. Have you enabled jumbo packets on your Ethernet card?

@LeeYangLBLBCS
Copy link
Author

LeeYangLBLBCS commented Aug 7, 2020 via email

@LeeYangLBLBCS
Copy link
Author

LeeYangLBLBCS commented Aug 7, 2020 via email

@MarkRivers
Copy link
Member

RX packets 1380807 bytes 12319408894 (12.3 GB)
RX errors 0 dropped 1001450 overruns 0 frame 0

Note that it has recevied 1.3M packets and dropped 1.0M packets. That means something is very wrong, and it has nothing to do with Spinnaker or EPICS. You need to figure out why the network card is dropping about 50% of the incoming packets. See if you can copy large files back and forth between that new system and another computer over the 10Gbit interface and get close to the full bandwidth.

Is your new system running the same version of Ubuntu as the old system?

I think the older spinnaker v1.20 has some problems with newer computer hardware.

That is very unlikely. At the APS we are running Spinnaker v1.20 on at least 5 different hardware configurations, using Ubuntu 18, Centos 8, and RHEL 8.

I installed newer spinnaker v2.0.
I can stream videos at 66 Hz with spinview.

If you put the camera in 8-bit mode with SpinView you should be able to stream 163 frames/s. Can you do that?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@MarkRivers @LeeYangLBLBCS