-
Notifications
You must be signed in to change notification settings - Fork 88
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ideas for updated CLI in conda-project
#362
Comments
Nice! I've been thinking about adding some optional arguments to
so it feels a little more like |
I really like that idea. Every new project now I do So I think it'd be awesome if Another feature would be an |
Another stretch goal would be to move config to |
Why not make |
Also, I do think the sub-command proposal above (Alternative 2) would indeed help tidy things up! |
With this proposal, I count 18 'top level' Here are the ones currently proposed without subcommands, though I have some suggestions for more subcommands which would bring it down to 16 commands total. I have also attempted to order these by how often I use them personally:
Proposed subcommands (also roughly ordered by my frequency of usage):
Of these, I could do away with Lastly, I never use |
Interesting perspective re: I think
|
Clean also removes downloaded data (defined in the |
The service feature was never fully developed and maybe it doesn't really need to be there. For example the conda package https://anaconda-project.readthedocs.io/en/latest/user-guide/reference.html#services |
And don't forget |
I concur, I think there's value in following conventions and usage from Given the plugin efforts in Originally I wanted to vote for the proposed alt 2 CLI (bc it followed the subcommand logic used by I would also like to see us including a |
+1 for |
What's the "m" stand for? I don't use So it sounds like I want the opposite of what some people here want -- I'd either want no circumstance where [ana]conda-project modifies the file, or for all such commands to be organized under an |
I think it's
|
"Project management" is an unfortunate pun, then, since this software is about projects but not about project management as the term is usually used! :-) |
@jbednar that is a fair perspective. I wonder if we could do something like a For me, I like being able to do When I start a new project using
That was my workflow every time I set up a new analysis in the past when I wasn't using P.S. in that case, |
Personally (as I've said already), I don't like the commands that edit the yaml (and sometimes the yaml gets completely reformatted, e.g when platforms are added after locking!) but I do know they have an audience. One proposal I remembered is an I think this could have a bunch of advantages:
|
Right, @mattkram, how you're using poetry represents the opposite user contract: The program provides ways to handle the typical modifications you'd want, and then you could edit the file, but are always suspicious that if you do so, you'll mess up the automatic editing somehow. That's an example of the program owning the config file, not the user, and the fundamental interface being a command-line rather than a text-file interface. The current anaconda-project is in a muddy middle ground, where many of the things you'd want to express aren't expressible with commands, and so my reaction to that is to consider the commands a distraction and not useful. But the opposite is also defensible, if the command interface is sufficiently expressive. I'm dubious about whether it could be made sufficiently expressive to cover how commands work in projects, though. |
I'll also add one more comment about alternative 1 versus alternative 2 described at the top of this issue: I think the most important thing is consistency within the conda/anaconda ecosystem. There may be counterexamples but commands such as |
I was thinking I personally feel pretty strongly about following in the footsteps of existing tools. Not necessarily because they do it best but rather to avoid having to learn yet another set of commands that effectively do the same thing just with a different backend. If we rely on I do agree that commands that modify the config should be very clear in that they do so and there should be a strict limited number of commands that make config modifications. But programmatically modifying the config should still be a first-class citizen. Config files are easy for us devs to look at and update but I'd strongly resist claiming that for all of our users. As a side note I've also found that any tutorial/docs that can specify running a command modifying the config to be (1) more concise and (2) easier to follow. |
Editing .yml files is a convention that follows existing tools, just different tools (notably conda). :-) I think anaconda-project does preserve comments, ordering etc., in the cases I've seen; the issue is just that a user is never quite sure whether that will really be true or if there are limits to it. In any case, yes, if modification were limited to Overall, yes, executing a simple command is more predictable and conveyable while being less expressive, thus more suited to a tutorial, while editing a file is more expressive and less conveyable and predictable, and thus more suited to advanced or daily usage, so they both have their uses if they can be kept distinct. |
One more point, which I think is one that fundamentally drives my support for an "automatic dependency tracking" feature. Nearly every time I am in a Jupyter notebook, I realize I forgot to add a package. I can do And why I really like |
Are you arguing that So if I'm understanding you correctly, there are actually then three (or more?) independent and orthogonal requests here for functionality already in anaconda-project that people want to have added to conda:
I'm hearing @kenodegard and @mattkram focusing on 3 while @jlstevens and I focus on 1 and 2, while conda-lock focuses on 2 (but without the notion of a command that runs in the environment). Personally, I think 3. is a valid request and a useful concept but not one related to projects, because to me a project has a specific stored and curated command that is meant to be run in the specified environment. So I'd argue that all three can be implemented separately in some sense, with CLI add/remove supporting either project or environment files, locking available identically for both bare environments and projects, and projects simply adding the notion that a command is being captured. If people agree with all the above, I'm still slightly unclear on how to achieve 3 without the notion of capturing a command, because to me the fact that an environment is tied to a specific command is what lets it be separate and not get polluted by my daily need to install stuff I need in my working session. I know how to curate environment+command as a unit, because to me that has a very clear remit: the envt contains only what that command needs, and no more or less. Each project is a separate environment from my daily working environment, which gets filled up with all sorts of non-reproducible who knows what as I do my work, and keeping projects fully separate like that is what makes anything reproducible for me. So, I'm not sure how to achieve 3 (curating a separate environment declaratively) without tying the environment to a specific command, but maybe I'm just showing my ignorance of poetry and other approaches. Probably something worth discussing orally instead of typing more here, though! Maybe it's a usage of |
Thank you for the write-up @jbednar, I think it is pretty close to my perspective as well. However, I tend to believe 2 & 3 should be integrated into a single solution, and then 1 is a follow-on from that. For me, I create a new isolated environment for every project (analysis, repo, application, library, etc.). I prefer to keep them in a local path rather than global, and I barely ever use Just to clarify my thinking, my understanding of abstract and concrete dependencies is this:
With the CLI, I can update/edit the abstract dependencies file, and automatically generate the locked dependency file and keep them in sync. I can use a pre-commit hook to ensure I never commit an unlocked or mismatched pair of files, such that I can completely tear down and reproduce the environment from every platform it is said to be supported on.
Whether these should be two separate efforts is an interesting question. I wouldn't be opposed to that, but I would say they should be designed to work together. My initial sense was that changing Wherever we end up, I think there are some amazing things related to environment management in |
Ok, sounds like we're on the same page, i.e. about having separate but mostly overlapping goals! |
I agree with @mattkram and @jbednar I see I absolutely love the commands feature. This is the part that makes a project a general purpose package to me. I see how lock files and environment management via Another thing to keep in mind, the |
True; both package recipes and project .ymls are specs for building artifacts, with projects being more general in that they encompass activities (like launching a live notebook) and not simply static outputs. But locked projects are also less general in a different sense, in that (unlike packages) they cannot be combined together.
Yep; having multiple commands that all share the same envt is crucial; we use that for all of the things you mention.
I think the independence only goes one way, i.e. that locking is useful for conda in general, while capturing commands without locking is only of limited use since it will soon break as dependencies change. So I would only argue that locking is valid on its own. add/remove, that's a separate story. :-)
I'm not sure how often that will come up, and getting a shared file format between conda and projects has proven to be a years-long endeavor, but sure; aim to unify all the ymls! :-) |
In preparation of upcoming hackdays, I'm collecting some ideas around how the CLI could be refined when converting
anaconda-project
to a potentialconda project
extension.This table summarizes potential renaming's of sub-commands, which would all be prefixed by
conda project
or `conda-project.After typing this, my personal preference is Alternative 2, where essentially many of the hyphenated subcommands are further broken down into higher level sub-commands such as
service
with operations likeadd
,remove
,list
, etc.If would also be great if somehow the
conda install PACKAGE
command could somehow be intercepted from within an activated environment to add toanaconda-project.yaml
using the current behavior ofanaconda-project add-packages
.The text was updated successfully, but these errors were encountered: