Build and link a local library in a Xamarin project - xamarin

Question

So I'm trying to automate the builds for a Xamarin project and integrate it to our company's CI pipeline. I'm not really familiar with .NET/Xamarin building process, but I've done it before following the really short guide for CI they have.
I'm using a standalone macOS build server (since we have to build iOS .ipa with XCode) with Visual Studio for this purpose.
Turns out, this time, we have a local dependency (some library we needed to slightly modify) integrated as a git-submodule on the project. The developers are all using the IDE for this and it loads/unloads and resolves all the dependencies without problem (or at least they told me that): they just have to hit the "play" button and it simply builds everything.
I'm trying to find a way to the same programmatically (using msbuild on the CI server) and here's where the problems start: so far, every time I run
$ nuget restore projectApp.sln
[...] # first get the project packages
$ msbuild \
/p:Configuration="Debug" \
/p:Platform="iPhone" \
/p:BuildIpa=true \
/p:IpaPackageDir="./Builds" \
/t:Build project.sln
... I get ~250 "are you missing an assembly reference?" errors. I assumed that meant the library dependencies were not found, so I tried resolving them with nuget:
$ nuget restore TK.CustomMap/TK.CustomMap.sln
[...]
But running msbuild again returns the exact same output:
[...]
Project "/Users/builder/project/projectApp.sln" (1) is building "/Users/builder/project/projectApp/projectApp.csproj" (2) on node 1 (default targets).
Project "/Users/builder/project/projectApp/projectApp.csproj" (2) is building "/Users/builder/project/TK.CustomMap/TK.CustomMap/TK.CustomMap/TK.CustomMap.csproj" (3:2) on node 1 (default targets).
CoreCompile:
[...]
"/Users/builder/project/projectApp.sln" (Build target) (1) ->
"/Users/builder/project/projectApp/projectApp.csproj" (default target) (2) ->
"/Users/builder/project/TK.CustomMap/TK.CustomMap/TK.CustomMap/TK.CustomMap.csproj" (default target) (3:2) ->
(CoreCompile target) ->
Api/Google/GmsDirection/GmsDirection.cs(2,18): error CS0234: The type or namespace name 'Http' does not exist in the namespace 'System.Net' (are you missing an assembly reference?) [/Users/builder/project/TK.CustomMap/TK.CustomMap/TK.CustomMap/TK.CustomMap.csproj]
Api/Google/GmsDirection/GmsDirection.cs(5,7): error CS0246: The type or namespace name 'Newtonsoft' could not be found (are you missing a using directive or an assembly reference?) [/Users/builder/project/TK.CustomMap/TK.CustomMap/TK.CustomMap/TK.CustomMap.csproj]
First of all, I want to know if there's a way to build the library fork on a different, internal pipeline and serve it + handle it in the project using nuget (like I'd do with the original version).
And then, in case that is too much over-engineering, how to properly build locally the whole project (including the library) and link it all together.