Lingering Issues for First Public Release

[josiahseaman]

Lingering issues:

• Endbin calculation is still overshooting
• Link Navigation is very precisely targetted
• test if always loading more than one chunk
• screen/scroll width reduced as well as expanded
• update on jsonName change
• leftXstart is not calculating X correctly when using "Show only Rearrangements"
• nucleotides look like they're shifted one to the left
• Component segementation should never combine bin 0 with another component
• Sometimes some nucleotides are not rendered
• consider whether to get a new bin range at each user input. if beginBin is 1 and the user wishes to enter beginBin 1000, unnecessary calculations will be started for beginBin 10 and 100.
• Consider to change 'Pangenome Bin Position' with 'Component range' when only rearrangements are shown.
• When only rearrangements are shown, fill the screen to avoid white spaces on the right.

From Dmytro's list

• "Save Image" button does not work.
• Jump to path at nucleotide position: when increasing the position spinner there's an exception "Uncaught TypeError: e.props.store.getPath is not a function at onChange (ControlHeader.js:191)"
• Jump to path at nucleotide position: select path, set position to some number, press "Jump" - there's an exception "Mixed Content: The page at 'https://graph-genome.github.io//Schematize/' was loaded over HTTPS, but requested an insecure XMLHttpRequest endpoint 'http://193.196.29.24:3010/MT233523/0'. This request has been blocked; the content must be served over HTTPS." URL.js:22 . Note the position in URL is 0, and not what was asked for
• Jump to path at nucleotide position: in Firefox (Linux) there's no combobox indicator for the "path" field.

From Simon's list

• For better orientation, it might help to display the pangenome length somewhere.
• Clicking on jump gives me:
  
• A very wide Column Width leads to ugly arrows:
  
• Hovering over a link, it might help to display the pangenome position or at least the bin I would jump to. That will affect the click decision.
• Use vertical compression leaves some grey space at the bottom which does not correspond to any path, I think.
• To have even better orientation in larger zoom levels, drawing the pangenome position from time to time on top of the components might help.
• npm start gives me a compilation warning: Browserslist: caniuse-lite is outdated. Please run next command 'npm update' <-- Updated the packages' json files

[AndreaGuarracino]

From a performance perspective, only the components whose start and/or end bins fall within the selected bin range should be rendered. New components on the right are rendered when their start bin is included in the range, and the components on the left disappear when their end bin are excluded. This lead to an up/down movement when the shift lead to load new rendered LinkArrows: it is not beautifull, but the performance are improving.

When only the rearrangements are shown, it is not clear how many columns to shift for each bin unit changes. Maybe the components should be entirely visualized or not

but the feedback is not good (you can't know how many time to click to hide a component because it is not visible how long it is). Perhaps in this display mode, the unit must become the component and not the bin?

[josiahseaman]

• leftXstart is not calculating X correctly when using "Show only Rearrangements"

"only the rearrangements" is a bug. It's not being calculated correctly in leftXstart. Component width calculated in the JSON "x" is basically useless in this mode, so ignore component.columnX. Instead, the padding for a whole chunk is JSON.chunk.component_count and JSON.chunk.link_count. For components inside of a chunk, the X will have to be manually calculated with a cumulative sum.

I was considering removing the "Show Only Rearrangements" checkbox for now until this gets fixed. It's better that every feature available is bug-free. Unless this bug is resolved quickly, we can just ensure it's not accessible today.

[AndreaGuarracino]

Now, when only rearrangements are shown, the X start should be correct.

I am still not sure how to shift in this visualization mode. When only rearrangements are shown, the component width does not correspond to the number of bin, so I can't shift it 1 bin at time. For now, it is avoided any shift and each component appears/disappears entirely when its EndBin is/isn't in the visualized bin range.

[AndreaGuarracino]

@josiahseaman, the implementation can certainly be improved, and will be done in the future, but I think the functional parts of the issue are done and ready for tests.

[josiahseaman]

I've now confirmed CS nucleotide slicing has one extra nucleotide. It just doesn't show up in small_test because the fasta file ends. If you add more, then you'll see it.

[subwaystation]

Great job guys. I am experiencing a much better browser feeling now!

I took some time and played with the tool. Here are some comments from my side:

• We may want to solve CS might add links where none are given from odgi bin component_segmentation#48 before a release?
• I pulled the latest code and did npm i which results in modified package.json and package-lock.json. Switching branches is not possible anymore, then. Do we need to push the current output of npm i or how can I circumvent this?
• I can't see any documentation about the fact that I would have to switch to the deploy branch in order to see any kind of visualization. But @6br and @subwaystation might be filling that gap with a tutorial.
• At a certain row height, displaying the path names on a fixed, foldable panel on the left would make a lot of sense to me.
• It might be that the current regex in the odgi server might not be able to deal with all path names. This is something for @subwaystation to check.
• I am missing a foldable summarization panel which displays e.g. the current number of paths in the view, the total number of paths in the data set, etc.
• To get a better feeling, where we actually are in our pangenome, I would suggest displaying a navigation bar. See the following example:
  
  As we are aiming for a COVID-19 Browser MT_350@NC_012920.1: 350 bp could be adjusted to display the information about the linear reference.
• The copy number legend is really nice, but does not provide any meaningful information for the COVID-19 data set? How about making it foldable? The legend's coloring seems to look the same as the component connectors. This might confuse people. This might be interesting for @urbanslug I think?
• Ideally, I don't have to click to see the next pangenome positions, but I can scroll through the pangenome. But I understand that this is not trivial to implement.
• Changing the name in the field on the left upper part besides Save Image does not do anything. We may want to put it to idle mode in the deploy branch.

Questions from my side:

• How would I use Schematize now, if I want to visualize my own data? Would the pipeline of @6br still work?
• Can we please have some very tiny default data people can browse in the master branch?
• Do we actually have an odgi server for the path index running somewhere?

I know it looks like a lot, but its mostly minor stuff. Anyhow, you really improved this a lot since the hackathon!!!

These are just comments and suggestions. I understand if we would not be able to solve all of them.

[AndreaGuarracino]

Navigating between very variable regions (full of arrows) often leads to this error:

[AndreaGuarracino]

Some news and proposals

To know always where you are (especially when only rearrangements are shown):

To know where a rearrangements starts/ends:

To know how long the pangenome is:

Bigger columns lead to bigger (and not ugly anymore) arrows and also bigger nucleotides:

[AndreaGuarracino]

Clicking too fast the navigation buttons (requesting too many operations) leads to incomplete rendering

I think the interface should be disabled until the current operation is complete (else, in the future, we should reimplement the architecture in a more robust way).

[AndreaGuarracino]

Now we can visualize the entire pangenome with bin width equals to 1, and it doesn't crash:

[6br]

For me I have some comments:

• The copy number legend is really nice, and I'd like to move the position of legend to right bottom or right top because left bottom is often overlapping components.
• It's better to show some messages when the file we specified is not available.

* I found minor thing when I set row number as 1.

• In some case, still I see warning message that Konva warning: NaN is a not valid value for "y" attribute. The value should be a number.

[6br]

I found that we cannot modify the end of the position. Why does it happen?
I recommend to set the background color to gray for easily knowing this field is read-only.

[AndreaGuarracino]

Initial implementation of a navigation bar:

[AndreaGuarracino]

Now the navigation bar is clickable to go to a specific region with a click.

[josiahseaman]

Wow! What an exciting feature! I think you should recruit a second person to test it, maybe Dymtro? Then we could merge it to master and deploy. I've been thinking about provenance coloring a lot lately. Being able to add in paths for gene annotations would be a game changer for our release...

…

On Tue, Jun 23, 2020 at 8:19 AM Andrea Guarracino ***@***.***> wrote: Now the navigation bar is clickable to go to a specific region with a click. [image: image] <https://user-images.githubusercontent.com/62253982/85415120-3a283e00-b56d-11ea-842b-1f5ac2550f9d.png> — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#86 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AARG2FEE3YKDNV46K7JAE2LRYC2X5ANCNFSM4NEH35OQ> .

[AndreaGuarracino]

@dimatr, I wait for you with open arms, destroy all my changes.

@josiahseaman, I want to point out that in my last commits I assumed the use of newer odgi versions, where there is a slightly change in odgi bin when the bin width is 1, so the dataset should be update before merge the last changes to master and deploy. The provenance coloring will be the next step (by the way, will it this the format for metadata?).