Thanks for getting this started! This is something the CLI team has wanted for a while now, but hasn't had the time to tackle themselves.

Your approach seems sound, and is a good start. Part of the reason why the team hasn't gotten around to this is the large number of edge cases it exposes. This is also why I've marked this PR as "breaking", and why this probably won't land until around the time npm@5 is ready to be released.

The most significant issue is that the DSL used for expressing SemVer ranges is inclusive enough that it will almost certainly collide with real repo commitishes (for instance, both 1.x and v2 are valid ranges, according to node-semver). With this approach, there will be no way for people to depend on branches or tags so named. I can't say for sure, but it seems probable that this will be a blocker for more than a few packages.

There seem to be a few ways to work around this:

1. Use an unlikely prefix on ranges in Git dependencies (semver-? I'm not attached to any spelling.), instead of using the heuristic of testing the fragment as a SemVer range. This is verbose, and doesn't match how Bower does it, but doesn't break backwards compatibility.
2. Add a way to "escape" fragments as literal commitish matches. This puts the work where it belongs, which is on the less common case.

On the whole, given the priorities of the project, #1 is better. Even with that approach, this is still a breaking change, because it changes how package.json is interpreted in a way that can only be used by new versions of npm, but it doesn't retroactively change existing package.json files.

What do you think?

(In addition, whatever we decide to do, the package.json docs will need to be updated to incorporate this, and something this edge-casey is going to need a bunch of tests (and maybe some spec documentation) so that those edge cases are hemmed in pretty tightly.)

Thanks for the reply! The current implementation definitely would break semver-valid branch names, in the case matching tags also exist. I did see some other suggestions in the various issue threads that I liked, which would solve the issue:

Currently the commit-ish is parsed out using normalize-git-url, which considers the commit-ish as everything after the #.

We could instead borrow from the syntax of npm install [<@scope>/]<name>@<version range> and use the @ delimiter to indicate we want semver.

• git://github.com/user/project.git#1.x does not break, and will checkout 1.x as it currently does
• git://github.com/user/project.git@1.x will evaluate to the semver range

How does that sound?

It has the same problem as my approach #2, in that it will cause all older versions of npm to choke on dependencies so specified. So, I don't think it's an improvement over using an unlikely prefix.

I would propose a third option:

1. If it is not a valid semver specifier, use it as a comittish as we do today.
2. If it IS a valid semver specifier, check to see if a tag or branch has an name that exactly matches it:
   a. If one exists, use that. Also check to see if any tag or branch matches it as a pattern. If there's a match there that is not the same as the exact match, warn.
   b. If one does not exist, use it as a pattern match against tag and branch names.

This makes the only scenario where we pick the wrong thing be:

1. We're using an ambiguous specifier (eg v1.x)
2. The package has both sorts of tags, eg v1.0.0, v1.1.0, v1.3.0, v1.x
3. The user entered git://foo/bar#v1.x and got the v1.x branch but actually wanted semver matching on the tags.

In that scenario we warn that we're picking the v1.x branch even though if it were a range we would have used v1.3.0.

Because this approach biases toward branches/tags, it won't step on anyone's existing use cases. And If you said v1.x but wanted it as a range it's easy for you as the person writing the package.json to use a different range specifier with the same meaning, eg ^1.

@othiym23 I agree it's not really an improvement on #2. I just thought it would be nicer syntax than a unlikely prefix, and would keep the semantics of the #commit-ish exactly the same.

@iarna I do like the idea of making this Just Work for the majority of use cases. But, my only thought against it would be the added complexity of the resolution logic and potentially more edge cases which could be introduced. Using some variant of #1 would make it simpler to reason about, and more explicit that you are choosing to resolve semver.

How much do we want to prioritize not breaking older versions of npm?

The way I see it option #1 and #3 would result in older npm versions throwing some variant of this error:

npm ERR! fatal: ambiguous argument '1.x': unknown revision or path not in the working tree.
npm ERR! Use '--' to separate paths from revisions, like this:
npm ERR! 'git <command> [<revision>...] -- [<file>...]'

Option #2 or some variant of it would result in more undefined behavior. A quick test results in something like:

npm ERR! fatal: remote error:
npm ERR!   user/project@v1.x is not a valid repository name

or when using user/project shorthand for github:

npm ERR! enoent ENOENT: no such file or directory, open '/home/sankethkatta/user/project@v1.x'

Looking through Bower's implementation, their logic goes like this:

1. If target is a commit hash, directly checkout the hash
2. If target is a valid semver range, create a list of semver-valid tags (by running semver.clean).
   a. If no tags exist, and target is *, default to master
   b. If a tag matches the target with semver.maxSatsifying, use this tag
3. If no tag matches the target with semver / target is not a semver range, look for an exact matching tag, or branch in that order

So Bower seems to do the opposite of the third option, it biases towards semver over exact tags/branches.

@othiym23 @iarna Any thoughts on the best option to move forward with?

@sankethkatta I think the ultimate consensus on the cli team was that using an unlikely-to-be-in-use prefix on a comittish part of the specifier is the right answer, eg #semver-^1.3.x or #range-^1.3.x. What exactly you use for that, I leave to you.

(Part of the thinking here is that npm really does work better with registry dependencies so while this is a use case we want to support, it's not a use case we want to encourage. As such having it be a bit ugly to use is perfectly ok.)

Sounds good, I'll move forward on implementing with an unlikely prefix

@iarna This should be ready to review. Let me know if I should expand on the documentation. I ended up choosing semver: as the unlikely prefix, it seemed to look the best with ^ and ~ ranges.

Bump, this is extremely important for corporate projects, relying on internal resources, that are updated regularly.

Any news here?

@StyleT we can't merge this until it's time for another major release, probably npm@5, but this is something we're definitely planning on including once it's time. I can't give you an exact timeline for npm@5, but the intention seems to be "in the next couple of months"

@zkat This is the best news I've heard all week.

@AngefloMusic Whether or not the feature's there, you might want to consider some kind of private registry setup. Git support can never be nearly as fast as registry installs are.

Should this now be closed due to #15666?

@ljharb indeed!

@sankethkatta this specific PR did not land because the changes that came with npm5 made the requirement completely different. That said, I linked this PR in the npm5 release notes, and attributed the feature to your name. Thank you for taking the time to figure this out and do the very important work of prototyping it and sussing out the specific API we needed for this.

This is a feature a lot of people are excited about, and I'm really happy it was included in the release. Thank you. <3

Thanks @zkat! Much appreciated! I'm excited to start using it out :)

Hey guys, so I'm a little confused now. Is it supported in npm5 or not and if so, how does it work?

@krlwlfrt yes this is now supported in npm5. However, the docs have not been updated.
@zkat should I make a PR with these documentation changes?

For now, here are the docs from my diffs reproduced below:

npm install

npm install <git remote url>:

Installs the package from the hosted git provider, cloning it with git. First it tries via the https (git with github) and if that fails, via ssh.

<protocol>://[<user>[:<password>]@]<hostname>[:<port>][:][/]<path>[#<commit-ish>]

<protocol> is one of git, git+ssh, git+http, git+https, or git+file. If no <commit-ish> is specified, then master is used. A semver range may be used in place of a <commit-ish> to resolve against git tags. In order to differentiate between a <commit-ish> and a range, the range must be prefixed by semver:. For example:

git://github.com/user/project.git#semver:^1.0.0

Git URLs as Dependencies

Git urls can be of the form:

git://github.com/user/project.git#commit-ish
git+ssh://user@hostname:project.git#commit-ish
git+ssh://user@hostname/project.git#commit-ish
git+http://user@hostname/project/blah.git#commit-ish
git+https://user@hostname/project/blah.git#commit-ish

The commit-ish can be any tag, sha, or branch which can be supplied as
an argument to git checkout. The default is master.

In addition, a semver range may be used to resolve against git tags. In order to
differentiate between a commit-ish and a range, the range must be prefixed by
semver:. For example:

git://github.com/user/project.git#semver:^1.0.0

Just to cross-reference with issue (#17190)

Typing npm install user/project.git#semver:^1.0.0 doesn't persist the semver range in the package.json file as I would expect it to; instead it references the repository.

So... if you were to re-install a project you'd get a different version from what you actually specified.

@sankethkatta where are you looking? I updated the docs for npm install and they're live at https://docs.npmjs.com/cli/install

@kruncher cool. Nice catch 🙆

@zkat Ah missed it on the npm install page. I was looking at the package.json docs: https://docs.npmjs.com/files/package.json#git-urls-as-dependencies and assumed it was missing in both places.

@sankethkatta aha, that's where that is! I'll totally take a PR to fix the latter, but I'd rather so the same kind of [#<commit-ish> | #semver:<semver>] in all the examples.

@zkat sounds good! PR'd over here: #17728

We use a private hosted gitlab-server and it doesn't work either with the latest npm or npm-canary. For example there is a tag on the Repo that ist name v0.1.0 and my url ist <url-to-git>#semver:^0.1.0. Shouldn't this match?

It's seems there is an issue with the semver feature using git urls, see #17621

It is working for me in npm v5.3.0

yarn will support semver on git repositories but without semver: prefix

to clarify: it resolves tags first, branches second, and at last it tries semver on tags and branches, that's why it isn't breaking change and one is able to resolve any tag/branch