Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rehost exiv2.org on pixls.us server #1890

Open
clanmills opened this issue Sep 3, 2021 · 56 comments
Open

Rehost exiv2.org on pixls.us server #1890

clanmills opened this issue Sep 3, 2021 · 56 comments
Assignees
Milestone

Comments

@clanmills
Copy link
Collaborator

clanmills commented Sep 3, 2021

We've had a generous offer from Mica and/or Pat to host exiv2.org on their server. Currently it's hosted on AWS and I pay about $20/month (depends on bandwidth). Let's move the domain by the end of 2021. We should wait until after v0.27.5 ships on 2021-09-30. Let's aim for November 2021.

I am the current registered domain owner of Exiv2.org and I will totally cooperate. I'm confident Nehal (@nehaljwani) will contribute to getting this done. I think we should investigate transferring the registration for exiv2.org to Mica somebody involved with pixls.us.

The future of exiv2.org should be discussed by Team Exiv2. I think it should remain because it has unique documentation and copies of every release of Exiv2 going back to 2004. Additionally, closing exiv2.org broadcasts an incorrect and very negative message about the status of Exiv2.

The effort involved to make a release on exiv2.org can be easily reduced. Today, I build on Ubuntu, macOS/x66 MinGW, Cygwin64 and msvc2019. This can be replaced by the nightly build when it is augmented with MinGW.

If Nehal, or another Team Member, is happy to take on "Release Engineering", I will cooperate with the effort to modify the current release procedures which are here: svn://dev.exiv2.org/svn/team/website

@kevinbackhouse
Copy link
Collaborator

My opinion is that we should stop building binaries for MinGW and Cygwin after 0.27.5. We have an Action which automatically builds binaries for Linux, MacOS, and Windows. Those are the major platforms, so I think that is sufficient. I did try to add support for MinGW and Cygwin, but couldn't get either to work.

@clanmills
Copy link
Collaborator Author

clanmills commented Sep 7, 2021

Well, thank you for trying to get MinGW to build. I agree 0.27.5 is the final release for which binaries will be built on the mac-mini build server.

I hope we find a volunteer to take over responsibility for release engineering. I will cooperate to make the appropriate (small) changes to the web-site and the release process .

@clanmills clanmills modified the milestones: v0.27.5, v1.00 Sep 9, 2021
@kmilos
Copy link
Collaborator

kmilos commented Sep 10, 2021

FWIW, I was behind the last two releases of the MinGW package for the MSYS2 repo - no problems building over there so far, but that's not really CI (although I use MinGW to build and debug locally both the main and 0.27-maintenance branches from time to time). Generally if both the Linux and Windows/MSVC actions are in good shape, MinGW should be as well.

Those are the major platforms, so I think that is sufficient.

Actually quite a few apps use the MSYS2/MinGW route to provide Windows support, so it might still be worthwhile to make sure it has a decent level of testing. But I agree, there is no need to release the MinGW binaries, they should come from the repo.

Side note: not sure about Cygwin, but the MinGW actions should be relatively straightforward using https://github.com/msys2/setup-msys2 - this is e.g. what both darktable and rawspeed use for their only Windows CI.

@clanmills
Copy link
Collaborator Author

I added both Cygwin and MinGW to Appveyor and they are used on the 0.27-maintenance branch. I don't know if they are deployed for branch 'main'.

I believe the MinGW build is popular with Qt users and we should continue to support it. Can we discontinue supporting Cygwin/64? I think so.

@clanmills
Copy link
Collaborator Author

@alexvanderberkel @postscript-dev @kmilos @kevinbackhouse @nehaljwani @hassec @piponazo @1div0 @dhoulder

Alison says the bill from AWS this month for exiv2.org was $35 (usually about $20). My guess is that server used more band-width to deliver v0.27.5 downloads. Com'on guys, it's time to deal with relocating exiv2.org

I think it would be useful to hold a team meeting on Zoom to discuss 2022. Exiv2 v1.00 does not appear likely. Perhaps we could begin a new series of dots v0.9, v0.91, .... Yet another v0.27 security "dot" v0.27.6 may be required.

I am willing to support/mentor a new release engineer. I'm happy to attend a team meeting and will only speak when asked a question.

@Puffincloud
Copy link

Alison here!! Be in no doubt the Credit Card paying AWS will be cancelled by the end of January so please deal with this ASAP
Thank you.

@nehaljwani
Copy link
Collaborator

nehaljwani commented Nov 20, 2021

Dear Exiv2 Team,

I've set up a demo at https://njw.exiv2.org/ . It uses Github Pages to host the content. We can update the artifact urls for *.zip and *.tar.gz files to link to the 'assets' uploaded on each official release on Github. We can do the same for https://pre-release.exiv2.org/ as well. Please check if this is acceptable and then we can retire the AWS instance and the only portion remaining will be the DNS domain ownership.

Please note that this does not include Redmine (https://dev.exiv2.org/). It is theoretically possible to generate static content for old URLs and host it as well, but I'll need a list of URLs for which this needs to be done.

As for Release Engineering - happy to take it up. I've access to Linux, MacOS, Windows and can also cross-compile for non x86_64 architectures if needed.

Cheers!
Nehal

@clanmills
Copy link
Collaborator Author

Looks like a miracle to me, Nehal.

@mica offered to host. Maybe he can run Redmine on his server and we give him a back up of the database. The URL of the redmine server doesn't need to have anything much to do with exiv2. exiv2-redmine.pixls.us (or something like that).

Very happy to hear that you're willing to do the releases. Let's discuss the scripts involved when the team decides to make a release.

@nehaljwani
Copy link
Collaborator

Could someone email me @mica's contact information?

@clanmills
Copy link
Collaborator Author

@nehaljwani I've sent Mica a private message on discuss.pixls.us and asked him to join the discussion here.

@clanmills
Copy link
Collaborator Author

I've pinged mica again: https://discuss.pixls.us/t/hosting-exiv2-org/28039 (I suspect you can't read my message).

Alex asked me to make you on "Owner" of Team Exiv2 on GitHub. I've done that. Not at all familiar with GitHub Teams. I'm not sure we use them. Anyway, you should be set up. And I've boosted your GitHub permissions from "Write" to "Admin" (the top access level). "With great power, comes great responsibility".

@paperdigits
Copy link

Hi, sorry for the delay. I spoke to @nehaljwani on matrix, and there were two things to host: the website and redmine. I had not previously heard about hosting redmine, and since it isn't actively used, I've asked @nehaljwani to try and make a static site out of it.

As for the website, we can host that on the pixls.us infrastructure if you'd like.

@clanmills
Copy link
Collaborator Author

clanmills commented Dec 3, 2021

@paperdigits Very pleased to hear your voice, Mica.

For sure @nehaljwani is has much greater knowledge of servers that I do. So, I totally defer to your's and Nehal's judgement about what should be done. I know nothing about setting up Redmine, however I'm sure it's fairly straightforward. When it's configured, it should be "read-only" to ensure that folks can't post new issues on the site.

There are team resources stored in svn as svn://svn://dev.exiv2.org/svn/team/ For example the team directory includes the code for the web-site, the Exiv2 logo graphics, and my book. That's there because it has always lived there and never been moved. If Nehal wants to relocate those resources to GitHub, that's fine with me. The only restriction is that we must avoid "bloating" the product download from GitHub.com/exiv2/exiv2.git. The GitHub Wiki is at: https://github.com/Exiv2/exiv2.wiki.git Maybe that's a suitable place. @kevinbackhouse is an engineer at GitHub and can probably advise.

The only other feature on my "wish list" is email addresses such as "clanmills@exiv2.org" for use by team members.

At a later time, let's discuss the registered owner of the domain. I'd like that to be transferred somewhere.

@kevinbackhouse
Copy link
Collaborator

My suggestion is to create a few new repositories in the Exiv2 organization. For example, the svn repository can probably be converted to git and pushed to a new repo in the Exiv2 org. Similarly, we could create a repo there for the website (using @nehaljwani's GitHub Pages work).

@clanmills
Copy link
Collaborator Author

Thanks Kev @kevinbackhouse.

Nehal @nehaljwani can you take the lead of this please and say what's to be done.

I'm happy to move stuff (team, book, web-site) from svn to git. It's on svn because it was born there and has never been moved. Time to change.

@nehaljwani
Copy link
Collaborator

I will look into this. Will post updates shortly.

@clanmills
Copy link
Collaborator Author

Thanks, @nehaljwani You are a splendid fellow. Very pleased that you are dealing with this. Thank You.

@postscript-dev
Copy link
Collaborator

@clanmills: Something occurred to me, the new manpage uses https://clanmills.com/Stonehenge.jpg as a test image. At the moment this version of the manpage is only on 'main' and few people use it, but once it is on https://exiv2.org/ this will change. While the file's location may bring more traffic to your website, all downloads would be at your cost. If you want to, you could add a copy of Stonehenge.jpg to https://exiv2.org/? If you decide to do this, then let me know the new URL and I will update the manpage references.

@clanmills
Copy link
Collaborator Author

Great idea, Peter. I'll move it. I don't run up bandwidth charges on clanmills.com.

Alison is dusting her rolling pin and looking forward to hitting everybody in 2022 about the web-site.

I love Alison. She doesn't have a rolling pin. We've been married for 47+ years. Photo at our son Jamie's wedding in 2019.
IDSA0489 CR2

@postscript-dev
Copy link
Collaborator

@clanmills

Great idea, Peter. I'll move it. I don't run up bandwidth charges on clanmills.com.

Perhaps it would help to add a reference to https://clanmills.com/ in the file's metadata. This would show where the image came from and point anyone interested to your site. Maybe adding other licensing tags would also be useful for you.

On a related note, when I was looking for a test file for the manpage, I was surprised to discovered that the Exiv2 logos on https://www.exiv2.org don't contain much/any metadata.

The big logo (which is the same as exiv2.png in git) has:

$ exiv2 --verbose --print a exiv2-logo-big.png
File 1/1: exiv2-logo-big.png
exiv2-logo-big.png: No Exif data found in the file
exiv2-logo-big.png: No IPTC data found in the file
exiv2-logo-big.png: No XMP data found in the file

and the small logo has:

$ exiv2 --verbose --print a exiv2-logo-small.png
File 1/1: exiv2-logo-small.png
Xmp.exif.PixelYDimension                     XmpText     4  1370
Xmp.exif.PixelXDimension                     XmpText     4  2401
Xmp.tiff.ImageWidth                          XmpText     4  2401
Xmp.tiff.ImageHeight                         XmpText     4  1370
Xmp.tiff.Orientation                         XmpText     1  top, left
exiv2-logo-small.png: No Exif data found in the file
exiv2-logo-small.png: No IPTC data found in the file

This is something that I think needs to be addressed by someone before the next release.

@clanmills
Copy link
Collaborator Author

clanmills commented Dec 30, 2021

I've updated the web site: https://www.exiv2.org/Stonehenge.jpg I submitted a change to svn: svn://dev.exiv2.org/svn/team/website r5321 I donate 100% of the ownership of the image Stonehenge.jpg to Team Exiv2. I don't remember why I started using it to discuss features of the exiv2 command-line tool with users.

That version of Stonehenge.jpg is almost the original. The only edit was the Caption "Classic View" (in Picasa).

$ exiv2 -pa -g Caption -g description https://exiv2.org/Stonehenge.jpg
Iptc.Application2.Caption                    String     12  Classic View
Xmp.dc.description                           LangAlt     1  lang="x-default" Classic View
$ 

I used an on-line service to design the logo and purchased the package of files. I've never thought about the metadata in those files.

I can't imagine anybody being interested in clanmills.com. Friends/family follow our trips. If I miss putting up photos for a day on vacation, I'll get email from somebody asking if we're OK! It's currently being totally re-engineered in JavaScript and the 80,000+ photos have been moved to Google Photos. The site has decreased from 10GB to 50mb. Instead of 2 million lines of script generated HTML, there are 300 lines of JavaScript. It's been a huge task to deal with this. I started in August and it'll be finished next month. Delighted by the result. It's clean, consistent, faster, smaller and provides an up-to-date user experience.

@clanmills
Copy link
Collaborator Author

clanmills commented Dec 30, 2021

I know very little about the use of metadata, so I don't know what metadata should be inserted into the logo files. The whole package is stored at: svn://dev.exiv2.org/svn/team/logo_files. I think I created Logo.graffle from the SVG for use in OmniGraffle (Eh? Why don't I use Adobe Illustrator for drawing?). You can modify them on SVN.

I think @nehaljwani is going to move the team resources (including the website) to GitHub, so it's better to wait for that transition.

@nehaljwani and @paperdigits. I've lost the plot with this project. Nothing seems to have happened. It's easier to forget about jobs that are finished, than to have lots on the TODO list. Let's get the web-site moved in January (2022).

540 rmills@rmillsm1:~/gnu/exiv2/team/logo_files $ find . -type f | grep -v svn
svg/SVGlogo.svg                    svg/SVGlogoAI.svg                  svg/Albertus_Medium.ttf 
svg/readme.txt                     EPSLogo.eps                        svg.zip
twitterheader.jpg                  400dpiLogo.png                     400dpiLogo.jpg         
65377083.pdf                       facebookcover.jpg                  Logo.graffle
SmallLogo.png                      SmallLogo.jpg                      icons/favicon-16x16.png
icons/favicon.ico                  icons/apple-icon.png               icons/apple-icon-144x144.png
icons/android-icon-192x192.png     icons/apple-icon-precomposed.png   icons/apple-icon-114x114.png  
icons/ms-icon-310x310.png          icons/ms-icon-144x144.png          icons/apple-icon-57x57.png
icons/apple-icon-152x152.png       icons/ms-icon-150x150.png          icons/android-icon-72x72.png
icons/android-icon-96x96.png       icons/android-icon-36x36.png       icons/apple-icon-180x180.png
icons/favicon-96x96.png            icons/manifest.json                icons/android-icon-48x48.png
icons/apple-icon-76x76.png         icons/apple-icon-60x60.png         icons/browserconfig.xml
icons/android-icon-144x144.png     icons/apple-icon-72x72.png         icons/apple-icon-120x120.png
icons/favicon-32x32.png            icons/ms-icon-70x70.png            SmallLogoBW.png
SmallLogoBW.jpg                    PSLogo.ps                          linkedinstandard.png
twitterprofile.jpg                 400dpiLogoCropped.jpg              400dpiLogoCropped.png
linkedinsquare.png                 facebookprofile.jpg                Exiv2-64x64.png
541 rmills@rmillsm1:~/gnu/exiv2/team/logo_files $ 

@nehaljwani
Copy link
Collaborator

nehaljwani commented Dec 30, 2021

Apologies for the delay, a few updates:

  • After spending some time, I was able to generate a static copy of the Redmine website (read-only). I've pushed the content at https://github.com/Exiv2/dev.exiv2.org. One can take a look at the commit history to find out how it was done. I've hosted it at https://dev2.exiv2.org/ using Github pages. @clanmills could you please test it a bit to see if there are any broken links that need to be fixed? 🙂 Once I receive a green signal, I'll proceed with moving it to dev.exiv2.org.
    ❗Note: The current size of the static content is ~1.6GB and Github hasn't complained (I think there is a soft quota of 1GB). If Github does complain in the future, I'll seek help from @paperdigits for moving the static content to the pixls.us server.
  • I've converted the team/website folder into a git repository (with the history intact), available at https://github.com/Exiv2/exiv2.org. It has a makefile, so my plan is to add a Github action by following https://github.com/marketplace/actions/github-pages-action to publish the content. If this approach creates hurdles, I'll simply generate the static content and push it to a different branch on the repo and serve the content from there.
  • The 'team' folder has more bonus content. @clanmills Could you please point out the remaining folders that need to be a repo of their own (I can combine multiple into one too)?
$ du -sh team/*
87M     team/XMPsdk
292M    team/book
88K     team/contrib
64M     team/drawings
115M    team/libraries
364K    team/license
2.6M    team/logo_files
3.0M    team/meetings
384M    team/releases
46M     team/rmills
1.9M    team/tuan
11M     team/website

@clanmills
Copy link
Collaborator Author

clanmills commented Dec 31, 2021

@nehaljwani Thanks. I'll go over this carefully today and give you an update. Here are my initial comments. Please don't feel daunted by the length of my comments. You're not far away.

  1. The website should be about 10m:
505 rmills@rmillsm1:~/gnu/exiv2/team/website $ cd /tmp
506 rmills@rmillsm1:/tmp $ svn checkout svn://dev.exiv2.org/svn/team/website
A    website/Makefile
A    website/master
A    website/master/Stonehenge.jpg
A    website/master/news.xml
...
A    website/exiv2.png
A    website/license.txt
A    website/README
Checked out revision 5366.
508 rmills@rmillsm1:/tmp $ du -sh website/
 22M	website/
507 rmills@rmillsm1:/tmp $ du -sh website/*
 16K	website/Checklist.txt
8.0K	website/Makefile
4.0K	website/README
 56K	website/bin
120K	website/doxygen
 16K	website/exiv2.png
3.1M	website/html
 20K	website/license.txt
6.7M	website/master
400K	website/var
509 rmills@rmillsm1:/tmp $ 

The equivalent in GitHub is 3.2G. So we're storing lots of stuff on https://github.com/Exiv2/dev.exiv2.org.git that doesn't need to be there.

552 rmills@rmillsm1:~/gnu/github/exiv2 $ git clone https://github.com/Exiv2/dev.exiv2.org.git
Cloning into 'dev.exiv2.org'...
remote: Enumerating objects: 14880, done.
...
Updating files: 100% (14199/14199), done.
553 rmills@rmillsm1:~/gnu/github/exiv2 $ du -sh dev.exiv2.org/
3.2G	dev.exiv2.org/
554 rmills@rmillsm1:~/gnu/github/exiv2 $ 
  1. The releases directory on exiv2.org are tarballs.

I think we can store them on GitHub using the releases feature on GitHub. In future this page https://www.exiv2.org/archive.html can be replaced by this: https://github.com/Exiv2/exiv2/releases

Every release of Exiv2 starting with v0.26 is already on GitHub, so the following should be added:

675 rmills@rmillsmm-local:~/gnu/exiv2/team/website/html $ ls -1 releases | grep -v 26 | grep -v 27 | sort --numeric
exiv2-0.10-win.zip
exiv2-0.10.tar.gz
exiv2-0.11-win.zip
exiv2-0.11.tar.gz
exiv2-0.12-win.zip
exiv2-0.12.tar.gz
...
exiv2-0.25-win.zip
exiv2-0.25.tar.gz
exiv2-0.3-win.zip
exiv2-0.3.tar.gz
...
exiv2-0.9-win.zip
exiv2-0.9.1-win.zip
exiv2-0.9.1.tar.gz
exiv2-0.9.tar.gz
676 rmills@rmillsmm-local:~/gnu/exiv2/team/website/html $ 555 rmills@rmillsm1:~/gnu/github/exiv2 
  1. Here's what's in team.
 87M team/XMPsdk           Remove. 
292M team/book             Keep.     That's the raw "code" for my book (mostly test images).
 88K team/contrib          Keep.
 64M team/drawings         Remove.   Graphics for GSoC 2013 and the LGM/Rio/2017 presentation.
115M team/libraries        Remove.
364K team/license          Keep.
2.6M team/logo_files       Keep.     Discussed above.
3.0M team/meetings         Keep.
384M team/releases         Remove.   Discussed above.
 46M team/rmills           Keep.     My "stuff".  I should find another home for this stuff.
1.9M team/tuan             Remove.   Tuan's "stuff". 
 11M team/website          Keep.     Discussed above.

You can remove XMPsdk and libraries. I was working with them in 2017 when investigating how to remove the xmpsdk/ directory from the code-base. XMPsdk has copies of different editions of the Adobe's code. libraries has copies of zlib and expat to be used when building the Adobe code (and openssl, libssh and curl). See libraries/ReadMe.txt

  1. Don't forget about testfiles.

The following repos has test images. svn://svn.exiv2.org/svn/testdata

We don't need to do anything quickly about this. Let's discuss it once we've dealt with the web site.

  1. Validating the website html.

website/Checklist.txt recommends

Validate uploaded webpages with W3C HTML checker # https://validator.w3.org

I usually skip this step (out of laziness/exhaustion). We should use this to "polish" the HTML to be as good as possible.

@nehaljwani
Copy link
Collaborator

Hello Robin, thank you for checking. I just wanted to clarify that https://github.com/Exiv2/dev.exiv2.org.git is the static content from Redmine, not the exiv2.org website, and while calculating the disk usage, you should skip the .git folder, otherwise you end up counting twice. The largest portion in the checkout comes from the various attachments in Redmine from issues/posts/wiki. As of now, GitHub hasn't complained so I am a bit optimistic that we can leave it as is.

Also, I can also put the entire team folder as a repository on GitHub, instead of separating each or a combination of them (i.e. defer that decision for later)

@clanmills
Copy link
Collaborator Author

The security alerts are valid and have been serviced. They are not a threat to Exiv2 in any way. I had exposed my Google Photos API key. I've modified the file involved and cancelled the key on google. I had accidentally revealed the ftp password to a web site I managed about 15 years ago. I've removed the script with that password.

@nehaljwani
Copy link
Collaborator

Wish you all a very happy new year!

I'm working from Sri Lanka at the moment.

Release archives for versions earlier than v0.27 have been uploaded at https://github.com/Exiv2/exiv2/releases

Will work on the website later today.

@clanmills
Copy link
Collaborator Author

Sri Lanka. You get about. I've had loads of notifications about the releases on GitHub. That's great.

When you hit the inevitable "oh crap, this script doesn't seem to work", please let me help. I inherited most of that code from Andreas and it works pretty well. When the releases are available on GitHub, we can throw away some of the pages and scripts.

@nehaljwani
Copy link
Collaborator

I've set up the GH action workflow with this commit: Exiv2/team@79b8ca0

At the moment, it deploys to web2.exiv2.org. Once we are happy with the content, we can switch exiv2.org to this.

Any change pushed to the team repo will trigger a deployment of the website automatically. To trigger it manually, one can visit: https://github.com/Exiv2/team.exiv2.org/actions/workflows/exiv2.org.yml and then click on 'Run workflow'

From a cursory look at the deployment at https://web2.exiv2.org/ , the 'book' and 'download' tabs need to be fixed.

@clanmills
Copy link
Collaborator Author

@nehaljwani This looks really good, Nehal. You're a very talented server engineer. A few comments:

  1. The web-page footer and "Preview" background.

That hasn't been automated. I documented a find command in Checklist.txt to manufacture the label. I manually edit exiv2.css to display "preview" in the background. This all works by hand. When automating, I'm sure there's a better way. For example, we need a little JS file (which is a copy of data in var/__version__), and included that at the top of every web page. When every page is displayed, the JS can dynamically create the page footers and appropriate back-ground (with/without Preview).

Please ask for help if you need it with the JavaScript. It'll be a few lines of code.

  1. Automatically triggering an update to exiv2.org

Great idea. Makes me nervous.

Maybe we can move to continuous integration and have frequent releases. We need a team meeting to discuss this. @alexvanderberkel Can you set up something. It's time to talk on Zoom.

  1. The "Archives" button on the downloads page need attention.

It should link to the GitHub Exiv2 releases and probably shouldn't be called "Archives"!

  1. About release.exiv2.org and pre-release.exiv2.org

If we release 'main' as Exiv2 1.00 (or more likely 0.28), it can go on release.exiv2.org. If we have more 0.27 dots (v0.27.6), we'll need dot-release.exiv2.org and dot-prerelease.exiv2.org

  1. Are you running Doxygen to create the API documents, or simply copying the generated HTML from elsewhere?

  2. Hosting on Pixls.us. Can you engage with @paperdigits about this. When that comes together, I'll have to apply to have the domain registration IP changed. I'd also really like to transfer the domain ownership to somebody else.

I'm really pleased to see this coming together. Thank You very much for your work.

@nehaljwani
Copy link
Collaborator

On the road right now, so won't be able to check everything. For point number 5, I'm using the steps that you provided to generate everything. Please see the changes in this commit: Exiv2/team@79b8ca0 . For point number 6, doesn't look like pixel.us is needed right now. I am using GitHub pages for everything.

@clanmills
Copy link
Collaborator Author

Let's talk on Zoom one day when it's convenient for you. I at home most of the time because of covid.

Sounds as though you are generating the documents with doxygen.

The reason to host exiv2.org on pixls.us is to web host exiv2.org without running up a bill on AWS. In future https://exiv2.org will be more-or-less a static portal that directs users to GitHub. If we discontinue https://exiv2.org, we could wrongly send the message that Exiv2 is going to die. That's not the case. Exiv2 is stronger today than it has ever been. The project's in good shape.

The LGM conference is to be held in Sri Lanka in 2022. Because of covid, it was cancelled in France in 2020 and 2021. If the conference happens and you're in Sri Lanka, perhaps you could give a talk about the project. I'll help you with slides and presentation materials. You don't need to talk about metadata as my book deals with that. I don't plan to travel to Sri Lanka because of the carbon emissions. I seldom fly any more.

@nehaljwani
Copy link
Collaborator

I'm in Sri Lanka only for vacation 🙂 Will be returning back to the States in a few days.

GitHub pages don't require us to provide any hosting. The idea is to test out web2.exiv2.org and dev2.exiv2.org and if all works well, point exiv2.org and dev.exiv2.org to these. It comes free for open source projects on GitHub. So everything I've been doing so far is to shutdown the AWS instance and not rely on it anymore, which is why we won't need to transfer to pixel.us.

@clanmills
Copy link
Collaborator Author

Ah. Are you saying that we can get the domain registration of exiv2.org changed to an IP address on GitHub? How convenient.

Enjoy your vacation. It certainly sounds as if you've almost got everything fixed. My objective is to stop the bill from AWS and it sounds as though we are almost there.

@nehaljwani
Copy link
Collaborator

The ownership of the domain is still with us on the pairdomains account. We just add CNAME entries to point whatever-we-want-or-nothing.exiv2.org to exiv2.github.io and GitHub takes care of the rest. https://pages.github.com/

@clanmills
Copy link
Collaborator Author

clanmills commented Jan 3, 2022

@nehaljwani This is really good. So, I think I need to:

  1. Go over the GitHub hosted web-sites and give you more feedback.
  2. Apply to ICANN to register a CNAME for exiv2.org

And you need to:

  1. Continue with your vacation.

Right?

@clanmills
Copy link
Collaborator Author

It's even easier. We just to update pair domains when we're ready. So, it's up to me to give you more detailed feedback.

@nehaljwani
Copy link
Collaborator

No need for 2. The Pairdomains account already allows us to do that, and it's already done. This is why the static content is already being served by GH pages. We don't even have to worry about HTTPS, Github takes care of it as well.

~/e/team.exiv2.org ❯❯❯ dig -t cname web2.exiv2.org +short
exiv2.github.io.
~/e/team.exiv2.org ❯❯❯ dig -t cname dev2.exiv2.org +short
exiv2.github.io.

@clanmills
Copy link
Collaborator Author

Right. I wondered how you created sub-domains such as pre-release.exiv2.org, web2.exiv2.org and others. You do that on Pairdomains. Got it. And you've used GitHub pages to store the content of web2.exiv2.org

So, my task now is to give you more feedback concerning https://web2.exiv2.org. Some polish and fixing and we will not need AWS. And we don't need @paperdigits to do anything at all. Very good. I'll give you more feedback.

@paperdigits During the BMFF project, there was a discussion about storing CR3, AVIF and HEIC files on https://raw.pixls.us. Roman (the libraw engineer in Moscow) objected insufficiently raw, or something. There was a suggestion to create a parallel site such as test-images.pixls.us. Any progress with that?

@alexvanderberkel
Copy link
Member

wow! This is great progress @nehaljwani and sorry for the delay of my answer @clanmills . Is there still the need for the meeting or something I can do?

@clanmills
Copy link
Collaborator Author

I don't think we need a meeting to discuss this project. Thanks to @nehaljwani, we're really on track to have exiv2.org being hosted on GitHub and we can close the AWS node.

I feel we need a meeting to discuss the plan for 2022. At the moment, we can't answer any of the following questions:

  1. Will there be another 0.27 "dot" in 2022?
  2. Will 'main' ship as v1.00 in 2022?
  3. Will 'main' ship as v0.28 in 2022?
  4. What are the objectives of v1.00?

In the office, the bosses (marketing) determine priorities and engineering are expected to deliver to schedule. That's not a viable approach in open source. We can only ask contributors:

  1. What is your availability?
  2. Do you have a project which you believe you can deliver in 2022?
  3. What resources/help do you need to make your delivery?

I don't expect anybody to make a list of projects that they would like others to deliver. The meeting is to audit our status and to determine a realistic delivery schedule. The alternative to a meeting is for the project to drift without objectives. If everybody agrees that drifting is the correct/only way forward then that's what will happen.

There are a couple of pinned issues #1462 and #1466 which should be updated following a Team Meeting.

@clanmills
Copy link
Collaborator Author

@nehaljwani Apologies for taking time to give you feedback. I posted a note to you on Monday 2022-01-10 to say that I had important domestic duties would consume me until at least 2022-01-21. It's 2022-01-28 and this task is now on top of the TODO stack.

There are really two different matters:

  1. What needs to be done to https://web2.exiv2.org before it can "go live" as exiv2.org?
  2. What needs to be done to the website repos: svn://dev.exiv2.org/svn/team/website

We can defer dealing with web-site until we agree to make a release. @alexvanderberkel Are you going to call a meeting to discuss Exiv2/2022?

About https://web2.exiv2.org.

1. Domain Registration
I'm really impressed that you are using the exiv2.org domain and GitHub storage. I believe I am the registered domain name owner and I'd like that to change. I believe the registration is provided by Pair Domains and the bill has been paid through 2024. When we change the domain owner, can we also change the billing address.

2. Content
At the top/html level https://web2.exiv2.org looks OK. Almost everything "further down is broken. For example: the "Book" tab (on the ribbon) is a 404: https://web2.exiv2.org/book/index.html

The "Downloads" page has a string download_table where a generated table should be visible. The directory website/master has the "raw" code for the web site and uses symbols abcde for placeholders. The placeholder has substituted by generated code. For example:

706 rmills@rmillsmm-local:~/gnu/exiv2/team/website/master $ grep __download_table *
download.html.in:__download_table__
707 rmills@rmillsmm-local:~/gnu/exiv2/team/website/master $ 

The code is generated by bin/download.sh:

709 rmills@rmillsmm-local:~/gnu/exiv2/team/website $ grep __download_table bin/*
bin/download.sh:table=__download_table__
710 rmills@rmillsmm-local:~/gnu/exiv2/team/website $ 

Here are the scripts that do the generation:

716 rmills@rmillsmm-local:~/gnu/exiv2/team/website $ ls -1 bin/*.sh
bin/archive.sh
bin/buglinks.sh
bin/conversion.sh
bin/download.sh
bin/exiv2-buildwinexe.sh
bin/exiv2-htmlman.sh
bin/exiv2-nikon-lensupdate.sh
bin/getLatest.sh
bin/release_info.sh
bin/sample.sh
bin/tagtables.sh
717 rmills@rmillsmm-local:~/gnu/exiv2/team/website $ 

There are about 400 substitutions that occur. Most of them are boiler plate:

718 rmills@rmillsmm-local:~/gnu/exiv2/team/website $ grep '__.*__' master/* | wc 
     405     426   16615
719 rmills@rmillsmm-local:~/gnu/exiv2/team/website $ grep '__.*__' master/* | head -10
master/archive.html.in:__doctype__
master/archive.html.in:__header4__
master/archive.html.in:__navbar__
master/archive.html.in:   __archive_table__
master/archive.html.in:__footer__
master/archive.html.in:__bootstrap__
master/changelog.html.in:__doctype__
master/changelog.html.in:  __header4__
master/changelog.html.in:    __navbar__
master/changelog.html.in:      <pre>__ChangeLog__</pre>
720 rmills@rmillsmm-local:~/gnu/exiv2/team/website $ 

3. download.html is very broken
404: "Download Source" button
404: "Download Archives" button

I think you can probably replace both of those with a link to the releases page on GitHub: https://github.com/Exiv2/exiv2/releases

The builds are totally missing! This is the code in download_table. I think you can copy it from exv2.org for v0.27.5.

We've agreed to discontinue providing binary builds with new releases as they are provided in future by the CI.

screenshot_66

4. Other Tabs Look Good

I've browsed the pages under the tabs "Getting Started", "What's New", "MakerNote", "Examples, "API" and "Manual". They all seem to be OK.

@nehaljwani Thank You very much for working on this. I'm looking forward to this project being complete and to seeing a future release of Exiv2 roll out without much effort on my part. Please be assured that I will help and support you to assume the roll of "Release Engineer" for Exiv2.

@nehaljwani
Copy link
Collaborator

nehaljwani commented Jan 31, 2022

Hello Robin,

the "Book" tab (on the ribbon) is a 404
404: "Download Source" button
404: "Download Archives" button

All of these issues have been resolved with these commits. I've added some automation to compute the contents of the required tables. Could you please check https://web2.exiv2.org/ again? Please note that the download and archive pages will look similar, but the links will point to artifacts on Github.

What needs to be done to https://web2.exiv2.org before it can "go live" as exiv2.org?

Just an update to the DNS record for exiv2.org and an update here. I can do it whenever you give the signal.

What needs to be done to the website repos: svn://dev.exiv2.org/svn/team/website

These have already been converted to Git (keeping history intact) and are available here: https://github.com/Exiv2/team.exiv2.org

When we change the domain owner, can we also change the billing address.

This is a small change, that can be done anytime, whenever the new domain owner is decided.

I manually edit exiv2.css to display "preview" in the background.

I am assuming you are referring to https://pre-release.exiv2.org/. I haven't done any work related to this yet. Is this still needed?

I posted a note to you on Monday 2022-01-10

Just curious, which platform was this on? I didn't receive any email/IM on riot.im.

Could you please also review https://dev2.exiv2.org/ ?

@clanmills
Copy link
Collaborator Author

@nehaljwani I can't track down my message to you on 2022-01-10. This is really odd. It was on GitHub, yet my last contribution to GitHub was on 2022-01-09.

Anyhow, I've looked at https://web2.exiv2.org/ and that's looking good and ready to "Go Live". My main concerns were the book and the downloads page. You've fixed them. You've done some very good work on the Archives pages to retain the appearance while linking the GitHub artefacts. Well done.

dev2.exiv2.org also looks good. The "Activity" tab is giving a 404. That's ugly and harmless. There is no activity! There's a 2020 issue there and the person who reported it also knows the fix. I think I'll open an issue on GitHub about that and share the users work-around.

We don't need to bother with the "Pre-release" CSS business until we make a release. A couple of lines of JavaScript will deal with that.

Right. I think we're ready to "Go Live" on this. Well done, @nehaljwani. I hope to get this done by the end of January and you've delivered. Great Job.

@alexvanderberkel Can you figure out the domain owner for the future and co-ordinate with @nehaljwani on the domain admin chores. I would like Team Exiv2 to have a Zoom meeting to discuss what can/will/might be delivered in 2022.

@nehaljwani
Copy link
Collaborator

I've turned off the AWS instance yesterday. All websites (except pre-release) are being hosted by GitHub now.

The last snapshot of Redmine is available at: https://github.com/Exiv2/dev.exiv2.org/releases/tag/2022.01.31

The last snapshot of the (entire) svn repo is available at: https://github.com/Exiv2/svn.exiv2.org/releases/tag/2022.01.31

I've also added: https://testdata.exiv2.org

I'll delete the AWS instance once @clanmills gives the signal to do so (which is necessary for the bills to cease).

@1div0
Copy link
Collaborator

1div0 commented Feb 1, 2022

Namaste Nehal.

Looks good.

Thank you so much.

@postscript-dev
Copy link
Collaborator

@nehaljwani: While looking for something in Exiv2's Redmine website, I noticed a couple of small things.

If I try to search on https://dev.exiv2.org/ then I get a GitHub 404 error.

Also, if I search in https://dev.exiv2.org/projects/exiv2 then I don't get any results - even for something common such as "the". The URL of the results page contains my search term but all the values in the page always reset to default.

This probably isn't used by many people but I think that the search is still a useful feature.

@nehaljwani
Copy link
Collaborator

nehaljwani commented Feb 1, 2022

@postscript-dev That's expected as it is static hosting. There is no server side computation, so search won't work anymore. It is not a full Redmine deployment, as it would require a dedicated compute resource. My primary aim was to simply keep all the existing URLs working so that if any URL was indexed by a search engine, it would still work and won't throw a 404.

If it is really needed, one can write a one time automation to copy all issues to GitHub and close them and then one can use the search powered by GitHub. Unfortunately, I don't have the bandwidth to take on that task right now, at least not until mid May.

@postscript-dev
Copy link
Collaborator

@nehaljwani: Thanks for the quick reply.

I rarely use Exiv2's Redmine website, so this isn't that important to me. Although the search is useful, unless there is a simple solution, I am happy to leave this.

@clanmills
Copy link
Collaborator Author

@nehaljwani I really don't think it's worth spending more time on Redmine. We moved on from Redmine when v0.26 shipped in April 2017. (it's hard to believe it's 5 years ago now). What you have done seems good enough.

I'd like to echo Peter K's "Namaste" to You. Great Job, Nehal. I'm sure the rest of the gang will give @nehaljwani a round of applause. @kevinbackhouse @alexvanderberkel @hassec @Puffincloud @piponazo @paperdigits @postscript-dev

I spent another 30 minutes this morning trying to find my note to you from 2022-01-10 (or possibly 2022-01-11). Can't find it. It's somewhere.

@alexvanderberkel
Copy link
Member

Absolutely great work @nehaljwani!

@clanmills
Copy link
Collaborator Author

@nehaljwani Some user feedback. No action required by you.

I find it impossible to clone the team resources over https (and therefore my book and bin directories).

/var/folders/7v/fb_nv0v142xfz_xy25rqjpf00000gn/T/smartgit7464908304192637754> git -c credential.helper=/Applications/SmartGit.app/Contents/Resources/credentials.sh clone -v --progress --branch main https://github.com/Exiv2/team.exiv2.org /Users/rmills/gnu/github/exiv2/team.exiv2.org
Cloning into '/Users/rmills/gnu/github/exiv2/team.exiv2.org'...
POST git-upload-pack (341 bytes)
POST git-upload-pack (252 bytes)
4986 bytes of body are still expected
fetch-pack: unexpected disconnect while reading sideband packet
early EOF
index-pack failed

I've cloned it with ssh by following this article: https://www.geeksforgeeks.org/using-github-with-ssh-secure-shell/

I believe it essential to use the username GIT on the git clone ssh command:

$ git clone ssh://git@github.com/Exiv2/team.exiv2.org

@kevinbackhouse kevinbackhouse modified the milestones: v0.28.0, Backlog Nov 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants