add_image_annotation only works https

[carissableker]

Hi,

I am trying to load a background annotation (image) to my network. Using

p4c.add_annotation_image(url="https://www.ucsd.edu/_resources/img/logo_UCSD.png", network)

works: the logo is loaded, I can also view it in the ImageManager, the output is as expected

{'edgeThickness': '1.0',
 'canvas': 'foreground',
 'rotation': '0.0',
 'type': 'org.cytoscape.view.presentation.annotations.ImageAnnotation',
 'uuid': '5105cc4c-9dcd-4121-bd22-4c7631652ed9',
 'URL': 'https://www.ucsd.edu/_resources/img/logo_UCSD.png',
 'shapeType': 'RECTANGLE',
 'edgeColor': '#000000',
 'brightness': '0',
 'edgeOpacity': '100.0',
 'contrast': '0',
 'name': 'Image 5',
 'x': '140.44609144134952',
 'width': '500.0',
 'y': '-87.37795634841291',
 'z': '1',
 'opacity': '1.0',
 'height': '100.0'}

using

p4c.add_annotation_image(url="http://www.ucsd.edu/_resources/img/logo_UCSD.png", network)
or (after downloading the image)
p4c.add_annotation_image(url="images/logo_UCSD.png", network)
or
p4c.add_annotation_image(url="file:images/logo_UCSD.png", network)
or
p4c.add_annotation_image(url="/absolute/path/images/logo_UCSD.png", network)

creates an empty rectangle in the network with the output:

{'edgeThickness': '1.0',
 'canvas': 'foreground',
 'rotation': '0.0',
 'type': 'org.cytoscape.view.presentation.annotations.ImageAnnotation',
 'uuid': '74c2b263-fcd4-4def-bf94-fdffad2b55fa',
 'shapeType': 'RECTANGLE',
 'edgeColor': '#000000',
 'brightness': '0',
 'edgeOpacity': '100.0',
 'contrast': '0',
 'name': 'Image 5',
 'x': '-28.22070147718013',
 'width': '100.0',
 'y': '-99.1472863452772',
 'z': '1',
 'opacity': '1.0',
 'height': '100.0'}

is missing the URL.

Using Cytoscape 3.10 or 3.10.2 with py4cytoscape 1.9.0 on Ubuntu.

Not sure if this is a local issue or not.

[bdemchak]

Hi, Carissa --

I tried this in the Cytoscape GUI ... and chose the background layer, which I assume you mean to do (??).

With a little work, I got Cytoscape to show a logo file (UCSD-Jacobs School) as a background annotation.

The URL I used was file:///C:/Users/bdemc/Desktop/UCSD-Jacobs.png ... note the "file:///" preceding the path.

Confession: I didn't use py4cytoscape to do this ... I directly edited the session file and reloaded the session. (If you'd like to know the exact mechanics, I can burden you.) Still, I'd expect the same result by using py4cytoscape.

What do you think??

[carissableker]

Hi @bdemchak,

Yes, using the GUI works fine (for the https URL, and certain file paths). I tested paths using the Image Manager. I can select the file in the file browser, or put in the "Image URL" field a URL or absolute path. However, I want to generate many networks by batch, and use p4ycytoscape to avoid doing it by hand.

Here my test code is here (also pasted below). In the list of URLs, I have noted which work in the GUI. Some of these are obviously wrong, but I tried everything...

The first URL works for me:

None none of the others do:

I expected the absolute paths that work in the GUI would work through py4cytoscape. To note also, the resulting output of annotation properties is also missing the "URL" attribute for all except the https URL.

I am running Python 3.13.2 in Ubuntu 20.04.6.

Code:

import networkx as nx
import py4cytoscape as p4c

p4c.cytoscape_ping()
p4c.cytoscape_version_info()

g = nx.Graph([(1, 2), (3, 4), (1, 4)])

for i, url in enumerate([
            
    # GUI (Image Manager - Image URL) - yes, py4cytoscape - yes
    "https://www.ucsd.edu/_resources/img/logo_UCSD.png",
    
    # GUI (Image Manager - Image URL) - yes, py4cytoscape - no
    "/home/cbleker/test/logo_UCSD.png",
    
    # GUI (Image Manager - Image URL) - yes, py4cytoscape - no    
    "file:/home/cbleker/test/logo_UCSD.png",

    # GUI (Image Manager - Image URL) - yes, py4cytoscape - no    
    "file://home/cbleker/test/logo_UCSD.png",
    
    # GUI (Image Manager - Image URL) - no, py4cytoscape - no
    "file:///home/cbleker/test/logo_UCSD.png",
    
    # GUI (Image Manager - Image URL) - NA, py4cytoscape - no
    "file:logo_UCSD.png",
        
    # GUI (Image Manager - Image URL) - NA, py4cytoscape - no
    "file:./logo_UCSD.png",
        
    # GUI (Image Manager - Image URL) - NA, py4cytoscape - no
    "logo_UCSD.png",
        
    # GUI (Image Manager - Image URL) - NA, py4cytoscape - no
    "./logo_UCSD.png",

]):

    network = network = p4c.create_network_from_networkx(g, title=f"test {i} - {url}")
    r = p4c.add_annotation_image(url=url,  network=network)
    print(url)
    print("\t" + "\n\t".join(f"{k}\t{v}" for k, v in r.items()))
    print("---")

[bdemchak]

Thanks @carissableker ... will look later today.

[bdemchak]

@carissableker Thanks, Carissa ... I spent some time with this and found (my) a very sloppy mistake, both in coding and testing. I think I have fixed it, but have tested only on Windows. It seems like you have a Mac/Linux machine. Would you mind testing it, too?

Here are examples of legitimate calls from Windows:

add_annotation_image(url='https://www.ucsd.edu/_resources/img/logo_UCSD.png')
add_annotation_image(url='https://www.ucsd.edu/_resources/img/logo_UCSD.png')
add_annotation_image(url='C:\\MyFiles\\logo_UCSD.jpg')
add_annotation_image(url='C:/MyFiles/logo_UCSD.jpg')

For you, I would expect something like:

add_annotation_image(url='/home/username/MyFiles/logo_UCSD.jpg')

If this doesn't work, I'll ask you to collect the end of a log ... we can cross that road if we need to.

For your test, please use the (unreleased) version in the 1.12.0 branch.

[carissableker]

Yes!

Updated which URLs work from my previous test:

    # GUI (Image Manager - Image URL) - yes, py4cytoscape - yes
    "https://www.ucsd.edu/_resources/img/logo_UCSD.png",
    
    # GUI (Image Manager - Image URL) - yes, py4cytoscape 1.12.0 - yes
    "/home/cbleker/test/logo_UCSD.png",
    
    # GUI (Image Manager - Image URL) - yes, py4cytoscape 1.12.0 - no    
    "file:/home/cbleker/test/logo_UCSD.png",

    # GUI (Image Manager - Image URL) - yes, py4cytoscape 1.12.0 - no    
    "file://home/cbleker/test/logo_UCSD.png",
    
    # GUI (Image Manager - Image URL) - no, py4cytoscape 1.12.0 - no
    "file:///home/cbleker/test/logo_UCSD.png",
    
    # (not expected to work)  
    # GUI (Image Manager - Image URL) - NA, py4cytoscape 1.12.0 - no
    "file:logo_UCSD.png",

    # (not expected to work)          
    # GUI (Image Manager - Image URL) - NA, py4cytoscape 1.12.0 - no
    "file:./logo_UCSD.png",
        
    # GUI (Image Manager - Image URL) - NA, py4cytoscape 1.12.0 - yes
    "logo_UCSD.png",
        
    # GUI (Image Manager - Image URL) - NA, py4cytoscape 1.12.0 - yes
    "./logo_UCSD.png",

Running in Linux (Ubuntu 20.04.6).

This is good enough for me 🙏

---

As an aside:

I do find a little strange that "file:/home/cbleker/test/logo_UCSD.png" and "file://home/cbleker/test/logo_UCSD.png" work in the GUI but not through add_annotation_image. The form of "file:/absolute/path/img.jpg" works for me when
I have a node table column with this style of image URLs, and then do a pass through style mapping for the visual property "NODE_CUSTOMGRAPHICS_1" using style_mappings.

Edit - I address this aside in my next comment

[carissableker]

Ah, there is still some mishandling.
For all three of the file URIs, py4cytoscape (1.12.0) generates the URL as

"file:/home/cbleker/test/file:/home/cbleker/test/logo_UCSD.png"

To make sure file URIs are treated correctly, _get_url_cmd_string
90da482#diff-5e685b0c181b4a6d87b69ad062b382d18e89c71f88639a5fe7520c55875e29e1R1039-R1047
should be updated along the lines of:

def _get_url_cmd_string(url, optional=False):
    if url is None:
        if optional:
            return ''
        else:
            raise CyError(f'URL or path to image file must be provided.')

    if re.search('^file:', url):
        return f' url="{url}"'  

    if re.search('^http[s]*://', url) == None:
        url = 'file:' + sandbox.get_abs_sandbox_path(url)
    return f' url="{url}"'

This assumes the file URI as url would be acceptable to Cytoscape.

Now I spent 10 min digging into what should be valid URI arguments...

Valid file URIs according to RFC 8089 (with examples here: https://datatracker.ietf.org/doc/html/rfc8089.html#appendix-B) should be:

file:/absolute/path/img.jpg (accepted by GUI)

and

file:///absolute/path/img.jpg (not accepted by GUI)

while

file://absolute/path/img.jpg

Should be invalid, but is accepted by the GUI:

If I understand the use of file URIs, then this is something that should be escalated to Cytoscape.

[bdemchak]

Hi, @carissableker ... thanks for checking up on this and for your good suggestions. There are lots of aspects to this, and I'll try to explain.

Interesting that you referenced the RFC ... I was thinking about re-reading it myself. Now I have.

First, it's completely fair to evaluate Cytoscape's behavior against standards. Note, though, that Cytoscape has relied on Java libraries for file I/O ... Since both Cytoscape and Java predate this RFC (and its predecessor), hopes for compliance must be tempered. Additionally, for most of Cytoscape's history, it has addressed only the local file system, so URL-formatted resources are a relatively recent concern.

Given this, I'm happy to forward this thread to the Cytoscape developers ... they're closer to the code and would have a more informed opinion.

Second, the sandbox concept needs attention here, especially since it's alien to Cytoscape. I suspect you're not using it, so I'll explain. The idea is that py4cytoscape code running locally should have a fighting chance to be easily portable to cloud systems such as Google Colab. In this configuration, Cytoscape runs on your local machine, and the py4cytoscape notebook runs in the cloud. Running other (perhaps unknown) authors' code presents security issues that could endanger your file system or present cross-platform (e.g., Mac vs Windows) incompatibilities. The sandbox makes it easier to create portable code, and py4cytoscape encourages this. In a word, the sandbox mechanisms need to be able to rewrite relative file paths so they reference a confined local folder common to the cloud system and local PC.

The quick fix you suggested violates sandboxing. (Sandboxing is responsible for the mishandling you identified.)

py4cytoscape assumes there are three kinds of resource identifiers: 1) URL including HTTP, 2) URL including HTTPS and 3) a local file name. A URL that includes file: is new to this discussion, though reasonable. It's likely that the best treatment would be to simply remove file: from the URL so that sandboxing can manipulate the file path in the same way as it already does.

Something like:

url = re.sub(r'^file:', '', url) # with credit to Copilot for saving me the trouble.

I'll try that and see where it gets us.

Third, it's true that the annotation CyREST interface uses the file: notation for image file names. While it may be sensible, I think it's completely anomolous relative to other CyREST interfaces. I don't know where it came from, and now I'll be asking the author. Should be an interesting explanation. (There are other things about the annotation interface I need to ask about.)

Comments are welcome! Thanks for engaging on this.

[bdemchak]

@carissableker I have filed a bug report suggesting that all Cytoscape file dialogs accept the file: notation, and that file open dialogs accept http: and https:, too. So far, this format appears to be accepted only by the annotation API you're using. It's a fair suggestion, though. We'll see ...

Thanks!