Multiple versions of XDM on same machine (or do I even need to?)

[Nos78]

Edit: Added a synopsis at the start, fixed a grammatical error and modified the title of the final paragraph

Synopsis

I guess I'm posing the following questions; all are relevant, but may become irrelevant to what I want to do, as you will see, depending upon the answer(s).

• Has the logic in how XDM uses chunks.db and stitches together the downloaded chunks changed with the major version increment (e.g. v7.2.11 doesn't work the same as v8.0.29 because of being forced to by changes to the underlying libraries requiring such, for example).
• If yes, can I run Java-based and DotNet based XDM, side-by-side, on same machine (eg v7.2.11 and v8.0.29)
• I think there's a use case that has never been considered - do you concur? Data migration in limited storage space situations!

Introduction
Okay, I've browsed the discussions and have not found anyone asking this question (apologies if it has been covered, I could not find it). I guess this hasn't been relevant until now, since 2013 XDM has been a Java based project, and v8 moves to DotNet, there's been no merit in having multiple versions. As a programmer experienced in both Java and DotNet, I could explore this question and answer it myself, but I'd rather hear the thoughts of the author! By the way, @subhra74 I must take this opportunity to congratulate you on a suberb product; one of the best download managers I've ever used. Having just gotten myself an old Windows Tablet (Linx 1010, win32 Intel Atom) I've plumped for the latest XDM, the DotNet based one and I think your solution to the Chrome Extension problem to be ingenious! Developer mode is something I would not have considered, given the potential risk that may be posed to the non-educated, yet it's so...simple! Your text is clear, your instructions dummy proof, and with the nature of Open Source (all extension code being unpackable and browsable, therefore checkable, in any case) a developer would be insane to try anything malicious and risk their reputation for momentary & temporary gain. Well done! If you need help going forward, I've some free time I'd be happy to assist...

Background to My Question
Anyway, that digression aside, my question comes from having v8.0.29 running on my tablet. With limited device storage (32GB total, and Windows 10 isn't exactly small...) I've got a 64GB memory stick plugged in and have specified that for both the Temp and the Downloads location in the XDM settings.

My main PC runs the latest version of the Java based XDM. I am wondering if I can install the latest DotNet XDM alongside the Java one, without issue? In theory this should be possible, however I'm wondering how this would play out in practise - the same App ID, registry settings, AppData location, are they all the same? If so, then what I want to do is not going to work without breaking my existing set up...

Why am I considering this? And what I plan to do
Now, I've downloaded a 32GB zip file using XDM v8 on my tablet. Unfortunately, there is not enough space on the USB stick to stitch together the chunks. I use 5G/4G+ network with a metered data plan for my home broadband, and I'm loathe to download the file again since I've already got it! I'm going to start a new download of the same file on my main PC, stop it immediately, and then copy over the contents of temporary folder for that download from my tablet's USB stick (IE the abcd123e-12xy-9876-etcetera folder) into the new download on my main PC, deleting the empty chunk and db, so I'm replacing it with the completed, unstitched download.

A note for the interested: This works, I've done it before (but not between different machines running different versions of XDM-before I put the USB stick into my tablet, I tried downloading another file that happened to be too large for the internal storage. I was not aware of the size of the download until it was too late. When this filled up, I changed the locations of the temp and download to be the D: drive, my USB stick, and copied the contents of the partially downloaded from C:...\AppData... into a new download of the same file located in the new D:\temp folder). Whilst it will have a different ID, it doesn't care what's in its folder, since the chunk.db.1 are all going to be correct, and it's the same target zip file (obviously closing XDM whilst I copy the already-downloaded data). The main PC, with terrabytes of space available, will happily stitch together the zip without giving that annoying "Disk full or read-only" message.

Do I even need to run the same version of XDM?
So, now I've detailed what I have done previously (albeit on same machine and the same XDM version), and what I plan to do this time, my first question is therefore, has the logic of chunks and chunks.db even changed? If its the same code from v7.2.11 and v8.0.29 then it won't matter, it will still work. What's caused me to worry is the change from Java to DotNet. The potential for two different underlying file system libraries causing changes behind the scenes... I don't want to risk my data. I guess I've always the option to duplicate what I've got, so copying off the stick and not working from the source directly, thus keeping it safe if it all goes wrong.

Now, if the answer to that first question is, yes, the logic has changed slightly, then I'll need to install v8.0.29 on my PC so that it is the same version as my tablet. Is that going to affect my existing installation of XDM? And if it is, then we're back to the question I posed at the beginning of this thread, can I have two versions of XDM on the same machine running happily alongside each other? - Again, for the interested third party, examples of breaking changes could be: we're using different IDs or something silly like INT sizes or little/big endian and not recognising the IDs created in a newer version of XDM but opened in an older version.

**Signing off (and possible feature request! And volunteering to look at it if you agree) **
This thread turned out to be rather longer than I anticipated - and I've posed more questions than it's title suggests! But I don't think I'm going to be the only user who will want to know whether they can safely move from v7.x.x to v8.x.x without breaking what they've already got. Whilst I doubt there will be many who would consider running both the Java and DotNet together if it turns out that moving from v7.x to v8.x is a breaking move, I'll gamble that I'm probably not in a set of one user, either... And that set of users who run into the issue that I have is going to be smaller still (needing to copy a partial download because you've ran out of space) it might prove useful in the future. It also sounds suspiciously like a use case you may have never even considered! Changing the location of the temp files whilst there are partial downloads still running, the app could have a migrate feature built-in, and this whole damn process would have been as smooth as specifying the new location and let the app do the rest... I'm happy to fork and work on that, if you're in agreement it might be useful!

Good luck and, once again, thanks for providing me with years of trouble free download management!

Nos
(Michael)

[Nos78]

Okay, answerng part of my own question:
For this, I'm assuming that user "username" is logged in on a machine running Windows 10 Pro (just my neuro-diverse mind being thorough in the detail... pedant and proud going on 45 years now 😄) - want to not put my username in expanding the %AppData% environment variable for my examples below...

Installing v8.0.29 when v7.2.11 exists
So, I went ahead and installed v8.0.29 from the Windows Store (side note, this irritates me! There are MSI installers for all the binaries except the one I want. Pain in the arse, I'm the kind of guy who likes to download once, install many. I've downloaded and installed on my tablet, now I'm forced to download again to install it on the main PC. I should be able to use the same MSI, but noooo, I'm forced to let Microsoft hold my hand and do the download dance all over again. Let's do the time warp again... it's just a click to the left, and a link to the right. Put your hands on your mouse, you bring XDM down twice. It's the Microsoft store that really drives you insane... let's do the download again. I said let's do the time warp again...

Okay now I've gotten that small rant out of the way; the Java-based app uses a different default location for its AppData, and thus the temporary data folder is different:

• %AppData%.xdman\
• C:\Users\username.xdman\temp

The DotNet app uses this default location:

• %AppData%\xdm-app-data\
• C:\Users\username.xdm-app-data\Data\temp

In the start menu, it seems searching "XDM" yields only the DotNet version. I cannot find the Java-based v7.2.11 (at least within the start menu). I'll have a look, see if I can't find it from the location I installed it at (I don't use C:\Program Files... for my programs, since I have a relatively small 120GB SSD for C: and a much larger second drive, 2TB hybrid.

I had a few active downloads left in XDM v7.2.11, and for the purpose of this discussion I deliberately paused them, since I don't mind losing whatever they where (I started them a while ago, and if I haven't gotten around to finishing them by now, they really don't matter all that much!)

But, for anyone interested, they're not available from the v8 interface (at least, not without migrating them from the old app location into the new one - that's more difficult than simply copy/moving the files over. You'd have to start new downloads for each, then find the right hash and copy the contents of each from the old folder, into the new hash ID folder. Since v8 has a clean slate with nothing running, it will be easy to find the new ID, since no folders exist yet in /temp! It's locating the old ones thats a little trickier, since you'd need to ensure the right target file to be able to kick off a new download, and ensuring the right partial chunks are put into the newly created folder (with it's different hash ID - the different name won't matter, so long as you're copying all the chunks and the .db files into this folder, it'll all just...work).

Logic for combining chunks
At first glance, the temp directory structure of v8 is the same as v7, so in theory I should just be able to copy my stuff in and see if they don't get stictched up.

[Nos78]

Okay, I now have enough information to basically answer my own questions (aside from the question I posed to the author RE: potential use case).

❓ **Can XDM v7.2.11 and XDM v8.0.29 exist on the same PC?

💡 Yes! The different %AppData% (actually, XDM is not using AppData, but rather stores it's data in the root of the profile, e.g. C:\users\username\XDM-Data-Goes\In-Here, and this applies to both Java and DotNet versions. The former is named .xdman and the latter is .xdm-app-data) identities for the Java vs DotNet apps, mean that if you have not altered the default location(s), then both programs data will remain intact. The install process for v8 does not uninstall any prior versions. For all intent and purpose, you can treat the DotNet migration, v8, as a completely new application that happens to have the same three letter acronym.

📝 I have been able to run both executables and verified that the prior existing downloads for Java v7.2.11 are intact and unharmed.

❓ Can we migrate a partial download that was downloaded in v8.0.29 to a new machine?

Yes, we can! Because of the above, I have not explored whether this will work with a download started in v8.0.29 nd being copied into v7.2.11 (I hazard a guess of yes. but don't quote me). This is what I did for my partial (completed but unstictched) download on my Win 10 tablet.

⚠️ Why am I needing to do this? I have a metered data connection, so I am loathe to download another 32GB when I've got the data! To recap, this use case has occured because I began a 32GB download of a ZIP file, and there is only 28GB free space left on the device once the download has completed. Therefore, instead of assembling the downloaded chunks, XDM gives the error Disk full or read only and stops.

The % bar reads 100% - this changes to 0% if you close XDM and restart, for example after logging off and on again. But the progress bar remains as a solid line. The download hasn't gone, don't worry. It's just not able to be assembled (and I guess the % is now reflecting the %age assembled).

💡 My solution to this, is to take the partial download from my Win10 tablet (helpfully for me it is already located on a 64GB USB flash drive, so I'm just going to eject and put in my PC. Then copy off the folder, and the flash drive can go back into my tablet. If anything goes wrong, I've got the original download parts safely tucked away on said flash drive) - so we will be starting a new download of the same target, on a different PC, and stop it straight away. This PC was running v7.2.11, but for simplicity and just in case, I upgraded to v8.0.29 for the purpose of this task.

💭 I wondered if I could do this same migration from v8.0.29 but using v7.2.11 on the destination PC. My head says yes, the library has changed from Java to DotNet but the logic of the program hasn't... But since the new XDM happily co-exists with the old XDM, I don't need to test this theory

So once we started and stopped the download and closed XDM: Oh, ensure we copy to another machine which has enough space to be able to stitch the download (the contents of the temp folder; NOT the actual folder, since XDM will generate a new hash ID for a new download)

We open the Temp location of XDM, and watch this folder whilst we start/stop a download of the same target file. We're watching for the newly created folder (don't worry if you don't do this, you'll just be making it slightly more difficult to work out which folder relates to the new download. That is unless its empty, because you've no partials currently, in which case, you're sorted!)

We start, and stop, a download on our new machine. Ensuring XDM is not running (remember to close it in the task bar too, not just the open app window. It likes to hide away in the task tray ready to interept your browser links, so close this too).

Delete the contents of this newly created folder, because we don't need those empty chunks or the associated .db files. We're replacing those with the ones from our completed but unassembled donor download! Again, it's the CONTENTS we're interested in, not the folder name. We cannot copy the folder directly, so rubber-band select all the chunks and the db files, then drag them over and into the new download's folder.

Specifically, we navigate to the new folder, and enter the folder created for the download (and if you're not sure which partial download you're searching for, because you've already got other partial downloads, you can open the .xdm-app-data\Data\ folder and open each of the .info file in Notepad.

Do not modify the .info contents, we're just using this to locate the download link and thus verify the name of the download.) Using the file name of the .info file, we can rescue and stitch together the download which is located in the folder of the same name as the .info - this name, btw, is an ID string or a Hash - I dunno which method he's using to generate (that's the abcdef-0123-456f-0987654fedc that the .info file is named as, as I say, that file has a correspondingly named folder with the same ID. You'll copy the CONTENTS of the folder, but not the folder itself, over into the new folder).

Once we've got the partial but unassembled chunks and db files in place in their new location, we go ahead and open XDM and resume the download. It will start assembling the chunks if you've done everything right!

Waayhay

[Nos78]

Marking this as the answer - all that remains is the question I pose to the repo owner, that is whether they think this use case(s) are worth pursuing, eg. adding migration when upgrading (or for that matter, between machines on same LAN?) or, as was my case, when we run out of storage space so that we have downloaded all the chunks but cannot assemble them.