Debugging a failing NuGet Package Cache task in Azure DevOps Pipelines

A little while back I was setting up a new Azure DevOps pipeline for a new project and noticed that the NuGet Restore task was taking more than half of the total pipeline time and so the ability to cache NuGet packages caught my eye.

Reading through the docs, it all seems incredibly simple - some minor changes to my csproj files to generate a lock file and a new task to add to my pipeline like so:

variables:
  NUGET_PACKAGES: $(Pipeline.Workspace)/.nuget/packages

- task: Cache@2
  displayName: Cache
  inputs:
    key: 'nuget | "$(Agent.OS)" | **/packages.lock.json,!**/bin/**,!**/obj/**'
    restoreKeys: |
       nuget | "$(Agent.OS)"
       nuget
    path: '$(NUGET_PACKAGES)'
    cacheHitVar: 'CACHE_RESTORED'

Simple right? It was looking good when I ran the pipeline for the first time, monitoring the build log in real-time as the new cache task detected a cache miss (as expected since this was the first run) and the fallback NuGet package restore kicked in.

That optimism was short-lived as the build failed with this obscure error on the automatically generated task that populates the cache with the restored packages:

...

There is a cache miss.

tar: could not chdir to 'D:\a\1\.nuget\packages'

Re-reading the docs to double-check that everything had been done didn't give any hints. Everything it told me to do I had done.

Turning to Google to try and find something to help pin this down turned up a very promising-looking GitHub bug report but reading through the comments and checking out some of the suggestions didn't make any difference.

A bit more searching turned up a different GitHub bug report about the NuGet cache task. Reading through that turned up this gem of a comment - basically, that NUGET_PACKAGES pipeline variable in the YAML snippet above is special-cased by the tooling to do some jiggery-pokery behind the scenes to override the location of NuGet packages. Quoting from that bug report:

If you use YAML, the NUGET_PACKAGES variable from the example will implicitly create an environment variable and override the location from ~\.nuget\packages to a the writeable location under d:\a\1\ -- so the fix here is to create a pipeline variable for NUGET_PACKAGES to point to the new location, and then your pipeline will pick it up and the cache task will work.

When I went back and looked at my pipeline, I had indeed renamed that NUGET_PACKAGES variable to something else. Changing that back to NUGET_PACKAGES and re-running the build, everything went green. Success!

It's annoying that the docs for the NuGet cache task make no mention of the fact that the NUGET_PACKAGES variable is special and must appear in the pipeline YAML exactly as shown in the documentation. Not being a fan of SHOUTY_VARIABLE_NAMES I'd renamed it to match the style of my other pipeline variables and broke things without knowing.

Having this information in the docs would have saved me a good few hours of debugging and research into what was going on. On the plus side, once it was all working, the time for NuGet restoration went from taking several minutes to 10s-30s so it did provide a significant speedup as promised.