Using relative url for git submodule needs credentials

Question

The Team Services documentation (https://www.visualstudio.com/en-us/docs/build/define/repository#what-kinds-of-submodules-can-i-check-out) points out that I can do a $ git add submodule if

it is an immediate submodule
Unauthenticated (n/a)
Authenticated
- Contained in the same team project
- Added by using a relative url from main repository

They give an example:

git submodule add /../../submodule.git mymodule

If I reference a git repo in the same project like

git submodule add ./../other-repo mymodule

It resolves the right repo but wants me to provide credentials. The build fails with the following message:

Cloning into 'mymodule'...
fatal: could not read Username for 'https://xxx.visualstudio.com': Invalid argument

Providing the full URL with credentials (https://user:[email protected]/...) works but is IMO a bad solution.

The documentation suggests that this should work with a relative url and without credentials. Am I wrong?

Edit 1:

Running with system.debug: true

Entering OnPrepareEnvironment
Primary repository: xxx
Calculating build folder hash key.
Loading tracking config if exists: C:\a\SourceRootMapping\07a8b96d-d805-4646-83d3-e7b2fbe394c2\18\SourceFolder.json
Creating new tracking config.
Loading top-level tracking config if exists: C:\a\SourceRootMapping\Mappings.json
Writing config to file: C:\a\SourceRootMapping\Mappings.json
Writing config to file: C:\a\SourceRootMapping\07a8b96d-d805-4646-83d3-e7b2fbe394c2\18\SourceFolder.json
Checking if artifacts directory exists: C:\a\1\a
Creating artifacts directory.
Checking if test results directory exists: C:\a\1\TestResults
Creating test results directory.
Creating binaries directory.
Setting local variables.
Create the initial timeline records for the tasks
Preparing repositories
repo clean = False
Found 3 endpoints to consider
Found 1 repositories to sync
Starting: Get sources
build.fetchtags=false
Entering GitSourceProvider.PrepareRepositoryAsync
Repository type=TfsGit
localPath=C:\a\1\s
clean=False
sourceBranch=refs/heads/r_080
sourceVersion=26d5a7a6e6ed47c8f12ee5dc5b376d6731b7863a
Syncing repository: xxx (Git)
repository url=https://xxx.visualstudio.com/_git/xxx
checkoutSubmodules=False
Starting clone
Checking out 26d5a7a6e6ed47c8f12ee5dc5b376d6731b7863a to C:\a\1\s
Checked out branch refs/heads/r_080 for repository xxx at commit 26d5a7a6e6ed47c8f12ee5dc5b376d6731b7863a
Leaving GitSourceProvider.PrepareRepositoryAsync
Leaving OnPrepareEnvironment
Running tasks
Starting task: Run git
##[warning]File name doesn't indicate a full path to a executable file.
Executing the following command-line. (workingFolder = C:\a\1\s)
git submodule add ./../other_repo mymodule
Error message highlight pattern: 
Warning message highlight pattern: 
C:\Windows\system32\cmd.exe /c "git submodule add ./../other_repo mymodule"
Cloning into 'mymodule'...
Fatal: InvalidOperationException encountered.
bash: /dev/tty: No such device or address
error: failed to execute prompt script (exit code 1)
fatal: could not read Username for 'https://xxx.visualstudio.com': Invalid argument
fatal: clone of 'https://xxx.visualstudio.com/_git/other_repo' into submodule path 'mymodule' failed
Finishing task: CmdLine
##[error]System.Exception: Task CmdLine failed. This caused the job to fail. Look at the logs for the task for more details.
##[error]   at Microsoft.TeamFoundation.DistributedTask.Worker.JobRunner.Run(IJobContext jobContext, IJobRequest job, IJobExtension jobExtension, CancellationTokenSource tokenSource)
Entering OnFinalizeJob
Leaving OnFinalizeJob

# Edit 2: I cannot answer this. The documentation is wrong/unclear, I went with the option to check out submodules like @eddie-msft said - but watch out: https://stackoverflow.com/questions/38752629/vsts-git-build-fails-with-git-submodule/

I tried with you command to add a Git Submodule and queue a build but didn't see any issue. Can you queue the build with "system.debug" variable set to "true" and then share the entire logs? — Eddie Chen - MSFT
– Eddie Chen - MSFT, Commented Aug 4, 2016 at 11:59
According to the logs, you are adding the submodule repo during a build process via "Command Line" build tasks. Can you try adding the submodule on your local machine and then pushing the changes? Please also try with "git submodule add ../other-repo mymodule" command. — Eddie Chen - MSFT
– Eddie Chen - MSFT, Commented Aug 4, 2016 at 12:58
I added it locally and tried to "git submodule update --init" in a build-step but that did just nothing. Using ../other-repo mymodule fails with the same error message. — kuechlerm
– kuechlerm, Commented Aug 4, 2016 at 13:06
You don't need to get it in build step namually, just check "checkout submodule" option in the build definition repository settings. — Eddie Chen - MSFT
– Eddie Chen - MSFT, Commented Aug 4, 2016 at 14:01

angularsen · Accepted Answer · 2020-09-25 16:55:45Z

6

I understand from the Submodules - Azure Repos that there are issues when:

Submodule repo requires authentication.
You have enabled Limit job authorization scope to referenced Azure DevOps repositories, which is enabled by default.

If you restricted the job access token as explained in the section above, then you won't be able to do this.

I have only tried nr 1 myself and it works for us.

Workaround 1: For submodule repo in DevOps

Disable Limit job authorization scope to referenced Azure DevOps repositories in project settings, which grants the pipeline access to all DevOps repositories.
Use relative paths instead of https:// in the submodule URL, such as ../MySubmoduleRepo or ../../../OtherProject/_git/MySubmoduleRepo.

Workaround 2: If different credentials are needed

Use a script to checkout with a personal access token (PAT)

Workaround 3: Custom script to reuse access token for submodule sync

https://stackoverflow.com/a/63593883/134761

steps:
- checkout: self
  submodules: false
  persistCredentials : true

- powershell: |
    $header = "AUTHORIZATION: bearer $(System.AccessToken)"
    git -c http.extraheader="$header" submodule sync
    git -c http.extraheader="$header" submodule update --init --force --depth=1

edited Sep 25, 2020 at 16:55

answered Sep 25, 2020 at 16:39

angularsen

8,7483 gold badges73 silver badges88 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Julien M Over a year ago

Using relative path fixed it for my azure pipeline! Thank you

Eddie Chen - MSFT · Accepted Answer · 2016-08-05 09:33:20Z

1

Please also check the path of your two repositories. If you have only two repositories in your project, the path of the first created repo A would be:

https://xxxx.visualstudio.com/_git/A

And the path of the second created repo B would be:

https://xxxx.visualstudio.com/A/_git/B

In this scenario, you need to use a relative path like following:

../../A/_git/B

edited Aug 5, 2016 at 9:33

answered Aug 5, 2016 at 9:24

Eddie Chen - MSFT

30k2 gold badges49 silver badges60 bronze badges

Collectives™ on Stack Overflow

Using relative url for git submodule needs credentials

Edit 1:

2 Answers 2

Workaround 1: For submodule repo in DevOps

Workaround 2: If different credentials are needed

Workaround 3: Custom script to reuse access token for submodule sync

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

Edit 1:

2 Answers 2

Workaround 1: For submodule repo in DevOps

Workaround 2: If different credentials are needed

Workaround 3: Custom script to reuse access token for submodule sync

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related