WebRAVE Curation Discussion

[shelbysawyer]

We will use this as a discussion spot for WebRAVE questions, struggles, breakthroughs, etc.

[lauren-herbine]

Hello all- thanks for a good first meeting! I am putting some notes from the "workshop" here for @Riverscapes/riverscape-project-curators to have as reference. I imagine updating this after each meeting so that we will have some documentation of tips and tricks we develop along the way.

WebRAVE Workshop Notes

9/9/2021

Cashe, Lauren, Wally, Shelby, Ally, Nick, Matt

Basic Steps:

(see more in-depth here)

1. Load project into Q

2. Export layer as GeoJSON

3. Open in Visual Studio Code (or other text editor)

4. Delete line starting with crs

5. Upload layer into Mapbox

6. Symbolize

7. Share -> draft -> download

8. Extract -> open style.json

9. Edit so that the text file follows this:

   1. {
        "$schema": "https://xml.riverscapes.xyz/Symbology/web/vector.schema.json",
        "legend": [
          ["hsl(Item1COlorHERE)", "WebRave Legend Item1"],
          ["hsl(Item2COlorHERE)", "WebRave Legend Item2"]
        ],
        "layerStyles": [
      	      PUT MAPBOX JSON HERE
        ]
      }
      

10. Save and commit in git

BRAT demo by Matt (random tips)

• cut-out a representative sample to decrease size for uploading to Mapbox

• style across data range -> filter

  • add stops to create bins
    

• can edit the steps/colors using json by clicking the <> button

• duplicate a layer by hitting the button next to the new layer button (+)

  • need new layers for each:
    • line
    • point
    • polygon fill
    • polygon outline
    • labels

• change the type to symbol if you want to add text (labels)

Helpful git discussions

• WebRave Raster symbolization method for categorical #130 (unresolved)
• WebRave Raster color ramps #115

Technical References

• WebRave Vector Symbology (out of date)

Other Resources

• Millimeter to pixel conversion
• WebRAVE staging server

WebRAVE Check-in 9/17/21

Lauren, Cashe, Matt, Shelby, Tanner, Wally
See recording for more in depth on these topics

Topics of discussion:

• styling line widths
• styling across the data range
• bringing in multiple copies of layers to style differently
• Tiling errors
• deadlines for different projects

Rasters

• issues with categorical data in rasters
• how to go from Q to WebRAVE by exporting color ramps
• put a note in pull requests that have .clr files for @MattReimer to convert them

WebRAVE check-in 9/23

Lauren, Cashe, Tanner, Ally
in person, so there is no recording

• Went through a couple of issues people are running into- came to the conclusion that they are all "back-end" issues and that we will continue to commit json files as we have learned to
• did an update on where everyone is at regarding their assigned projects. Sounds like everyone is fairly confident on meeting the end of September deadline

[lauren-herbine]

@Riverscapes/riverscape-project-curators here is the recording from the first "workshop"

[lauren-herbine]

@Riverscapes/riverscape-project-curators here is the recording from the second meeting (I started recording ~5 minutes in):

[MattReimer]

Awesome! Thanks @lauren-herbine

@Cashe93 (and everyone else) I know I said just commit the .clr files for discrete raster symbologies but I just ran through it and it's super easy so I updated the documentation on how to do it:

https://github.com/Riverscapes/RiverscapesXML/blob/master/docs/Symbology/webRAVERasters.md#palletted-unique-colors

See the section on "Palletted unique colors"

I really don't mind if you commit the CLR file but if you want to take a stab at converting it to the proper .txt that's good too.

[CHafen]

@Jdgilby check out the links on this thread to learn about curation for webRave

[Cashe93]

@lauren-herbine, did you say there is a link for the warehouse where we can access webRAVE projects? If so, I can't seem to find it.

[lauren-herbine]

@Cashe93 yes! It is under "Other Resources" as WebRave staging server, but here's the link too: https://northarrowresearchlabs.github.io/riverscapes-staging//#/Anabranch

[Cashe93]

Awesome! Thanks, @lauren-herbine!

[Cashe93]

@Riverscapes/riverscape-project-curators is there a way to include line widths in the JSON legend?

[MattReimer]

Just to make sure I understand:

You can have dynamic line widths using mapbox styling.

If you mean line widths represented in the simple legend in the white box on the right of the screen then no. Currently it's just color and label

[Cashe93]

Yeah, I was inquiring about representing those widths in the simple legend. Good to know, thank you!

[lauren-herbine]

October 15th Update

1. November 2nd: VBET and RSContext should be ready for full deployment

We will need eyes on the WebRave versions of both of these.

Open up the projects in Arc/Q and compare in WebRAVE. Have a fly around. Open up the different project views. Zoom in and out. Add all layers. etc.

• Is there missing symbology?
• Is there symbology that's not working the way it's supposed to be?
• Is symbology duplicated anywhere (ie in the shared folder and in a project folder)
  etc.
  @shelbysawyer (VBET) and @Tanmanfalcon (RSContext), can you take the lead on these? These need to be ready for public eyes/use by November 2nd for the Riverscapes Monitoring Summit. The rest of the curators can help out as well, but I think you all should have the final look over as you two have been working on these projects.

2. November 10th: BRAT

The same goes for BRAT, by November 10th. @Cashe93, are you the one who has been curating the BRAT projects? Can you take the lead on this? All of the above applies.

3. December 1st: Everything

All projects (Channel Area, Confinement, TauDEM, etc.) need to be ready for the public by the 1st of December. We will go through everything we will do for VBET and RSContext (see above). I will make sure all of our projects have review by the curation team.

Projects:

VBET- @shelbysawyer lead- Nov 2nd
RSContext - @Tanmanfalcon lead - Nov 2nd
BRAT- @Cashe93 lead - Nov 10th
Confinement- @lauren-herbine lead- Dec 1st
Channel Area - @lauren-herbine lead - Dec 1st
TauDEM- @shelbysawyer lead - Dec 1st

**Please create tickets noting any of these issues. At the minimum include a url to the staging server WebRave project you're looking at. Even better would be a screenshot(s), and the best would be a ~30 sec video showing what is going on. Assign these to Matt. **

Again, this should definitely be a group effort. If you're leading a project, use GitHub to assign reviews of HUCs that are on the staging server (not all need to be reviewed, but we should definitely look at a wide variety of riverscapes for each project). Create issues, and please reach out to @MattReimer quickly if you're running into problems. He'll be working on this for the next 2 weeks and wants to hear about things we're having troubles with ASAP! He will be working on getting the platform deployed by the end of next week, so we should have all major bugs to him/discussed with him by mid-week next week. We can then focus on tidying up the symbology (easier to push in patches even after the platform is deployed).

[CHafen]

Thanks @lauren-herbine. I don't have a lot of experience with WebRAVE, but I should be able to review some HUCs and make sure they look good. If any of the project leaders want to assign some HUCs to me I'd be happy to look over them. I am experience a bit of a problem though. I can't figure out how to access the Riverscapes Warehouse. I'm not sure if I accidently made two accounts and am signed into the wrong account or if I haven't been given access. But, anyway, how do I request access to the Warehouse @MattReimer ?

[MattReimer]

Let me try this again. @CHafen I have added you to the Staging server for riverscapes.

https://northarrowresearchlabs.github.io/riverscapes-staging//#/Anabranch

Let me know if you have any more problems connecting.

[CHafen]

Thanks @MattReimer looks like a can access the warehouse now.

[CHafen]

@shelbysawyer , @Tanmanfalcon @Cashe93 do you need any help reviewing any HUCs for the projects you're over? If so, I am available to help review them as needed.

[Cashe93]

@shelbysawyer , @Tanmanfalcon @Cashe93 do you need any help reviewing any HUCs for the projects you're over? If so, I am available to help review them as needed.

@CHafen, I'm good for now but will definitely reach out if anything comes up! Thank you!

[lauren-herbine]

Hey @Riverscapes/riverscape-project-curators and @MattReimer - I have a question concerning project views in WebRave. In Channel Area, the default project view is the same layer symbolized differently and displayed twice (example from Q):

It is the same layer (channel_area), just symbolized once as a hollow polygon and secondly as a filled polygon. There are two different symbology .json's associated that they call. The BL for the project view looks like this:

<View name="Channel Area" id="DEFAULT">
  <Layers>
    <Layer id="channel_area" />
  </Layers>
</View>

Question- in WebRAVE, only one of these layers will show up, and I can't figure out how to get the project view to include both versions of the layer.

Is there something I can change in the BL to make this work? The XML validator isn't letting me add the symbology tags to differentiate the two layers.

Thanks!

[MattReimer]

Ah, I see the problem. both your BL layers have the same ID. Make one of the ids different and then include the second layer in the view and that should do it.

<View name="Channel Area" id="DEFAULT">
  <Layers>
    <Layer id="channel_area" />
    <Layer id="channel_area_hollow" />
  </Layers>
</View>

[lauren-herbine]

Great, thanks! I just tried that

[lauren-herbine]

@Riverscapes/riverscape-project-curators You all may have already known how to copy and paste symbology between Q projects... but I didn't! And in case you also didn't know how to do that, here's a lil vid:

[joewheaton]

Pretty cool eh?

[shelbysawyer]

Hey @Riverscapes/riverscape-project-curators, I'm stuck on getting the symbology worked out for the Catchment Wing Stream Order vector layer in WebRAVE VBET. This 1-minute video covers what I'm getting stuck on:

Any advice is greatly appreciated, even if its just pointing me to a .json for a layer with similar complications!

[lauren-herbine]

@Cashe93 have you already symbolized flowlines in mapbox to correlate (as closely as possible) to what we already have in Arc/Q? I was just about to do the Mapbox part of this for ChannelArea, but if you already have the file done, could I use that?

If that wasn't clear- I'm trying to get this symbolized in Mapbox:

is this what you have already done?

[shelbysawyer]

@MattReimer I've got a few more specific questions now:

1. In QRAVE, the color coding goes by 1-6 and then the 7th class is "all other values". In Mapbox, I can't figure out how to get that 7th class to apply. Right now, I just have that class set to stop 7. Do you know how to address this?

2. For the different dot and hatch fills, I'm not sure how to get them in Mapbox. I can see the "fill pattern" box, but I can't figure out what it wants as an input.

[MattReimer]

Hey @shelbysawyer, I think you are on the right track. I could be wrong, but in order to mimic what's displayed in QRAVE, I believe you'll need 4 layers (hatch fill, color fill, outline, and labels). I had a similar issue with the flowlines.json since each line type had different patterns (and even then, I still wasn't able to recreate it exactly.)

Cashe is right. You will likely need separate layers for colors and for fill pattern

The real issue here is that we may need to create or upload symbols somehow so that our system can have access to them on our app.

I think @shelbysawyer for now maybe just make a note about this and move on. We can definitely support custom fill patterns eventually but I don't think it's going to happen for the first pass.

[MattReimer]

1. In QRAVE, the color coding goes by 1-6 and then the 7th class is "all other values". In Mapbox, I can't figure out how to get that 7th class to apply. Right now, I just have that class set to stop 7. Do you know how to address this?
2. For the different dot and hatch fills, I'm not sure how to get them in Mapbox. I can see the "fill pattern" box, but I can't figure out what it wants as an input.

Ah, ok. I think I see the problem here. @shelbysawyer your data source seems to be categorical right? Like, there's no stream order 1.1223123 right? In that case you should use "Style across data conditions" instead of "Style across data range". When you do that Mapbox gives you a fallback value to use in every other case

As for the hatching. I think let's just file a feature request and move on. We can definitely get there but I think version 1 just won't support it for now. Maybe use the color layer for the colors, leave the hatching layer out and we can come back to it.

[shelbysawyer]

@MattReimer Okay, I'll leave the hatching out for now and just do the pull request that has the other 3 style layers. I'll start an issue for the hatching background. Thanks!

[Cashe93]

Yes @lauren-herbine, the flowlines have been symbolized. However, I based it off of BRAT and there was no Canal/Ditch (Aqueduct) category, instead, there is only Canals/Ditches, so we may need to add that? Also, I collapsed all of the pipe categories into 1 instead of 6. You can view it on webRAVE here by adding the network layer. It might need some work so take note of any improvements you want to see and I can make the changes.

[lauren-herbine]

Awesome, thanks @Cashe93 ! Collapsing all of the pipes makes sense, since I am using the same symbology for all of them.. (brain fart moment). I don't think I necessarily need aqueducts split out from the rest of the canals/ditches. Your symbology looks good and I think will work well for Channel Area in WebRAVE. Can you drop those in shared?

[Cashe93]

No Problem. And the symbology should already be in the shared folder. Its called flow_lines.json

[lauren-herbine]

Great, thanks! For breadcrumb purposes: I copied @Cashe93 's flow_lines.json and then renamed it to FLOWLINES.json, as that is what ChannelArea requires. Both are in shared

[CHafen]

DELETED MESSAGE

@CHafen this thread is primarily for WebRAVE stuff. I moved your questions about QRave to its own thread (albeit kind of clumsily. Github doesn't have good message board tools).

Here it is:

• QRave Stretched symbology rasters don't transfer from Arc #310

[CHafen]

Also, hillshade and DEM don't have this problem. Not sure why they work and others don't...

[MattReimer]

Hey @Riverscapes/riverscape-project-curators just a heads-up: I rebuilt all the tiles on the weekend.

For most people you're going to see probably NO difference on rasters but vector layers should be working a lot better. They should:

• Look much better at lower Zoom levels
• Have a lot more geometry included (especially if you were seeing a lot of missing shapes before)
• NOT have reach gaps except in extreme cases

The downside is that you will notice some shape changing as you zoom in and out. This is because we can't keep everything perfectly at lower zoom levels so the algorithm approximates and simplifies.

I tried out as many projects as I could but it would really help me if you could each take a look at your respective projects and report back if anything is wonky.

Thanks!

[Cashe93]

@joewheaton, @MattReimer, and @wally-mac,

After a few small fixes, BRAT is now working across all three platforms of RAVE (Arc, Q, and Web). The only missing symbology is Flow Direction in WebRAVE (see #322 ). I've opened up a handful of projects in each platform and everything else seems to be in working order. I encourage @Riverscapes/riverscape-project-curators to open up some projects and look for issues that I may have missed, but as of now, things are checking out. See the following for screenshots:

• Test sqlBRAT in QRAVE #323 : QRAVE
• Test sqlBRAT in ArcRAVE #324 : ArcRAVE
• Test sqlBRAT in WebRAVE #300 : WebRAVE

[MattReimer]

@Cashe93 this is great. Those tickets with all the screenshots are awesome and really useful for proving/comparing where we're at now.

[CHafen]

@Cashe93 I noticed a few of the projects aren't working in WebRave. When I go to the Warehouse some of the projects say WEBRAVE on the right hand side above the date in orange. At a preliminary glance these ones seem to work. But some don't say WEBRAVE and these ones aren't working when I look at them

The ones that don't say WEBRAVE the capacity legend show up, but the network is just black

https://northarrowresearchlabs.github.io/webrave-staging/#/Anabranch/2da0b2d7-c975-4955-994a-849188c726c9

[Cashe93]

@CHafen, thanks for pointing those out! I've already informed @MattReimer about those projects. For some reason, those runs have 'Null' values for most of the Output and Intermediate attributes which is why they aren't displaying/being symbolized properly. Luckily it's an issue with those specific projects and not an issue with symbology or business logic.

[lauren-herbine]

@Riverscapes/riverscape-project-curators I've been looking around in WebRAVE for RSContext, and noticed that the symbology is coming in very wonky (opaque green HUC with red flowline network). I'm guessing this is not what it is supposed to look like:

Is this being looked at? Should I start an issue? Just not sure if it is something broken or if someone is actively working on the symbology and it is just not ready yet. Thanks!

[lauren-herbine]

Project I was looking at

[wally-mac]

@Cashe93 can you please look into this?

[Cashe93]

@wally-mac, on it. See below.

[Tanmanfalcon]

@lauren-herbine You're right, it's not supposed to look like that. There are about 4 or 5 vector layers (including the perennial network) and all of the watershed boundaries that haven't been displaying properly. I noticed the problem but haven't had the time lately to look into the problem or create an issue. I can work on it tomorrow, but if it is pressing you can go ahead and create an issue. The symbology should already be created, but again I haven't looked into if I did it wrong or not yet.

[Cashe93]

@lauren-herbine and @Tanmanfalcon I'm working on this as we speak. It looks like a handful of symbology was never created and/or never pushed to git. @leallysmith and I are currently putting together a list of the layers that still need symbology. Hopefully, I can get most of these done by the end of the day. @Tanmanfalcon, if the symbology has been created for all RS_Context layers, some of them still might be saved locally? If this is the case let me know ASAP before I get too deep into this.

[lauren-herbine]

Great, thanks for getting on this everyone! @Cashe93 @leallysmith

[Tanmanfalcon]

@joewheaton, @MattReimer, and @wally-mac,

RSContext is mostly working across all three platforms of RAVE (Arc, Q, and Web). There appears to be no missing symbology, but a few color ramps are displaying incorrectly in QRAVE and WebRAVE, discussed in the issues below. I will try and get those fixed or mentioned today. I've opened up a handful of projects in each platform and everything else seems to be in working order. I echo what @Cashe93 said about BRAT earlier and encourage @Riverscapes/riverscape-project-curators to open up some projects and look for issues that I may have missed. See the following for screenshots:

• Testing RSContext in ArcRAVE #335 ArcRAVE
• Testing RSContext in QRAVE #330 QRAVE
• Testing RSContext in WebRAVE #327 WebRAVE

[Cashe93]

@Tanmanfalcon, do you have a working list of the issues that need to be addressed? If you need help getting those fixed, let me know ASAP.

[Tanmanfalcon]

The biggest issue I've ran into so far is the precipitation color ramp is displaying incorrectly in both Q and Web. Since Web is just using the .txt symbology files that we created from Q, I'm guessing that the problem originates in Q. Simply using the SLYR tool to convert the .lyr symbology from Arc into .qml isn't working for some reason.

[wally-mac]

@Tanmanfalcon let me know if and how I can help you troubleshoot this.

[CHafen]

#310
See this issue. I noticed this as well and I'm not sure how to fix it.

[MattReimer]

Hey folks. webRAVE is now in production so we can use the production URLs from now on:

I wrote a little post about it here:

• WebRAVE Launched to production!! (and URL Change) #367

[wally-mac]

@MattReimer WebRave worked really well in the workshop today. Way to go. Congratulations

[CHafen]

Hey @Riverscapes/riverscape-project-curators I'm trying to get GUT symbology working on webRAVE. It's my first time doing symbology for webRAVE, so I'm following along with the videos @lauren-herbine posted from the webRAVE workshop. But when I tried to upload my GEOJSON files to mapbox I got an error as shown in the screen shot. I'm not familiar with mapbox and have no idea what this means or how to fix it. Anyone know what I might be doing wrong or how to fix this?

[lauren-herbine]

Hmmm... I am not familiar with this error- it looks like it has something to do with the spatial extent of your project. The only thing I can quickly think to troubleshoot is to ask if you deleted the line starting with "crs" in the geojson (using visual studio code or the like) before uploading to mapbox? I have a feeling that's not the problem, but just something to check.

You could also try clipping a piece of your map in Q or Arc to export a much smaller spatial extent- perhaps that would help?

[MattReimer]

I think what may be happening here is you're trying to upload projected data instead of WGS84.

If you reproject you should be able to upload. Mapbox doesn't do projected data.

[joewheaton]

So this thread is great (19 answers and 40 replies), but lets start new discussions from here on out if you have WebRAVE discussion. This one is so long it has lots it utility @Riverscapes/riverscape-project-curators ;)