I am running the below task to Upload Notebook to Databricks with Azure DevOps release pipeline:
databricks workspace mkdirs //build
databricks workspace import --language PYTHON --format SOURCE
--overwrite _databricks/notebook/$(notebook_name)-$(Build.SourceVersion).py //build/$(notebook_name)-$(Build.SourceVersion).py
but getting the error: Path (//build/helloworld-04747aa082426141c5c286fbe5eab2d7d1bd5983.py) contains empty name
2020-05-16T18:54:01.5665867Z ##[section]Starting: Upload Notebook to Databricks
2020-05-16T18:54:01.5831890Z ==============================================================================
2020-05-16T18:54:01.5832257Z Task : Bash
2020-05-16T18:54:01.5832539Z Description : Run a Bash script on macOS, Linux, or Windows
2020-05-16T18:54:01.5832833Z Version : 3.163.2
2020-05-16T18:54:01.5833073Z Author : Microsoft Corporation
2020-05-16T18:54:01.5833436Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/utility/bash
2020-05-16T18:54:01.5833824Z ==============================================================================
2020-05-16T18:54:01.8378449Z Generating script.
2020-05-16T18:54:01.8525502Z [command]"C:\Program Files\Git\bin\bash.exe" --noprofile --norc -c pwd
2020-05-16T18:54:01.8915864Z /d/a/_temp
2020-05-16T18:54:01.8975224Z
2020-05-16T18:54:01.9001870Z ========================== Starting Command Output ===========================
2020-05-16T18:54:01.9009310Z [command]"C:\Program Files\Git\bin\bash.exe" --noprofile --norc /d/a/_temp/5c6b246a-d6b4-45f3-9c35-c1f64320d54b.sh
2020-05-16T18:54:09.0829994Z **Error: b'{"error_code":"INVALID_PARAMETER_VALUE","message":"Path (//build/helloworld-04747aa082426141c5c286fbe5eab2d7d1bd5983.py) contains empty name"}'**
2020-05-16T18:54:09.1242892Z
2020-05-16T18:54:09.1369153Z ##[error]Bash exited with code '1'.
2020-05-16T18:54:09.1385662Z ##[section]Finishing: Upload Notebook to Databricks
Surprisingly, the same code was running fine without any error for other peoples.
Nothing found from google search. Any idea/clue/suggestion? Thanks
You are experiencing this error message due to incorrect path specified i.e. (//build). Make sure you pass the correct path i.e. (/build).
I would suggest you to correct the task to Upload Notebook to Databricks with Azure DevOps release pipeline as shown below:
databricks workspace mkdirs /build
databricks workspace import --language PYTHON --format SOURCE --overwrite _databricks-example/notebook/$(notebook_name)-$(Build.SourceVersion).py /build/$(notebook_name)-$(Build.SourceVersion).py
I had faced similar issue like mentioned in -: https://forums.databricks.com/questions/39420/path-contains-empty-name.html
This is because you might be using windows hosted agent. This can be solved by changing agent to use ubuntu. I used ubuntu 1604 (ubuntu-16.04).
Please use the path as mentioned below (no need to add "/" infront)
databricks workspace mkdirs /build
databricks workspace import --language PYTHON --format SOURCE --overwrite _databricks-example/notebook/$(notebook_name)-$(Build.SourceVersion).py /build/$(notebook_name)-$(Build.SourceVersion).py
I Hope this helps you solve it. Cheers!
Related
After Running this command on Azure Cli:
az vm extension set --publisher Microsoft.Compute --version 1.8 --name CustomScriptExtension --vm-name avmUat --resource-group argUat --settings '{"commandToExecute":"powershell.exe Install-WindowsFeature -Name Web-Server"}'
I get this error
Deployment failed. Correlation ID:
8d6972da-6d46-42e8-a053-f8e899348893. VM has reported a failure when
processing extension 'CustomScriptExtension'. Error message: "Invalid
handler configuration. Exiting. Error Message: Error converting value
"{commandToExecute:powershell.exe Install-WindowsFeature -Name
Web-Server}" to type
'Microsoft.WindowsAzure.GuestAgent.Plugins.CustomScriptHandler.PublicSettings'.
Path 'runtimeSettings[0].handlerSettings.publicSettings', line 5,
position 102.".
Edit:
The result when I run it in powershell on Azure Cloud Shell:
Bash and this is on bash (Gladly it worked)
It seems that these commands do not work in the Azure CLI locally on the windows command prompt. You should run these on a bash environment.
With the same Azure CLI version. The same commands could work in Azure cloud Shell but not local Azure CLI in command prompt. I also tried running it on local Azure CLI 2.0.58, still failure.
Moreover, Here is the same Sample script, which is running in a bash environment.
Run this on my local command prompt.
Run it on Azure Cloud Shell
I am learning something about ElasticSearch Stack and I am having a problem installing Logstash on Windows 10 (windows 10 enterprise N OS build 15063.674).
I installed ElasticSearch and Kibana and these are up and running.
I followed the steps on this page to install Logstash:
Step 1: Download and unzip Logstash
downloaded "logstash-5.6.3.zip" file and unzipped it to: "c:\program files\elastic\"
Step 2: Prepare a logstash.conf config file
as described here, I created a "logstash-simple.conf" in the "c:\program files\elastic\logstash-5.6.3>" folder
Step 3: Run bin/logstash -f logstash.conf
at this point I am having the issue (I tried using both cmd and PowerShell with elevated privileges): the result is:
The system cannot find the path specified.
"could not find jruby in C:\Program Files\Elastic\logstash-5.6.3\vendor\jruby"
Of course, the "vendor" folder exists, and there is a "jruby.bat" file inside. I searched the web and I found something about the JRUBY_BIN environment variable but event after the creation (and the additional reboot) the issue still is there.
Can someone address me to the problem?
I have found this solution: https://discuss.elastic.co/t/logstash-does-not-start-says-could-not-find-jruby-in/113500.
You can also try move logstash folder out of program files direct to C, it might help.
I tried to create a directory junction on Docker Windows to create 8.3 . But it failed with the error message below.
D:\data\docker\sample>docker build -t sample .
Sending build context to Docker daemon 1.272GB
Step 1/4 : FROM microsoft/windowsservercore
---> 2cddde20d95d
Step 2/4 : RUN fsutil.exe behavior set disable8dot3 0
---> Running in ec6e7cc09ec6
The registry state is now: 0 (Enable 8dot3 name creation on all volumes).
---> 53805bc21858
Removing intermediate container ec6e7cc09ec6
Step 3/4 : RUN mklink /J "C:\PROGRA~1" "C:\Program Files"
---> Running in 2116f2070e6a
Junction created for C:\PROGRA~1 <<===>> C:\Program Files
re-exec error: exit status 1: output: time="2017-10-03T16:15:54+09:00" level=error msg="hcsshim::ImportLayer failed in Win32: The system cannot find the path specified. (0x3) layerId=\\?\C:\ProgramData\Docker\windowsfilter\5234d5f7cd7c2669db1818d9227a4be2822eeb72fc30071a495de78cd578b9f3 flavour=1 folder=C:\WINDOWS\TEMP\hcs088640839"
hcsshim::ImportLayer failed in Win32: The system cannot find the path specified. (0x3) layerId=\\?\C:\ProgramData\Docker\windowsfilter\5234d5f7cd7c2669db1818d9227a4be2822eeb72fc30071a495de78cd578b9f3 flavour=1 folder=C:\WINDOWS\TEMP\hcs088640839
The Dockerfile was below:
FROM microsoft/windowsservercore
RUN fsutil.exe behavior set disable8dot3 0
RUN mklink /J "C:\PROGRA~1" "C:\Program Files"
CMD [ "powershell" ]
How can I create a directory juction on Docker Windows?
I'm guessing your issue is actually not related to creating a directory junction (note that your build did complete that step), but is related to exceeding the base size of docker containers (I see you're pulling from windowsservercore, which is about 10GB, and web browsing tells me the default container base size is 10GB).
Try changing the size to 20GB in the docker daemon config by adding this json:
"storage-opts": [
"size=20G"
]
I was experiencing the same error message, tried all manner of troubleshooting, and resolved it with that daemon setting.
For Docker Enterprise this can be configured here C:\ProgramData\docker\config\daemon.json
"storage-opts":["size=20G"]
If this file does not exist created it
Link here
I was not able to resolve this with other methods; what worked for me was deleting and re-pulling the base image. So, it is apparently possible to get this error from a corrupt image.
check if the project is Running or not ! if it is running finish it. then check the command again
It seems that junctions and and hardlinks are not supported by docker for windows.
I had similar problem trying to create a container with Cygwin inside. Cygwing uses hardlinks a lot. It all went wwll until Docker tried to commit an image and it failed with "hcsshim::ImportLayer - cannot find the path" error. When I got rid of the hardlinks in the container I was able to commit the image without problems.
I am following the getting started guide in the URL below
http://hyperledger-fabric.readthedocs.io/en/latest/
I installed all prerequisites.
Using windows 10.
After installation of prerequisites I move to next section “Write your First Application” URL given below
http://hyperledger-fabric.readthedocs.io/en/latest/write_first_app.html
After download “fabric-sample” from git when I try “startFabric.sh” script in “fabcar” subdirectory for launch the network I am encountering an error are show below.
UTC [main] main -> ERRO 001 Cannot run peer because cannot init crypto, missing /etc/hyperledger/fabric/C:/Program Files/Git/etc/hyperledger/msp/users/Admin#org1.example.com/msp folder
If you look at the path of the missing file you'll see it is malformed. It was resolved by issuing an export command before running the startFabric.sh script:
export MSYS_NO_PATHCONV=1
./startFabric.sh
See https://jira.hyperledger.org/browse/FAB-5392?page=com.atlassian.jira.plugin.system.issuetabpanels%3Aall-tabpanel.
Thanks Tim & Gari
I am creating Windows VMs from the azure xplat cli, using the following command:
azure network vnet create --location "East US" testnet
azure vm create --vm-name xplattest3 --location "East US" --virtual-network-name testnet --rdp 3389 xplattest3 ad072bd3082149369c449ba5832401ae__Windows-Server-Remote-Desktop-Session-Host-on-Windows-Server-2012-R2-20150828-0350 username SAFEpassword!
After the Windows VM is created I would like to execute a powershell script to configure the server. As far I understand, this is done by executing a CustomScriptExtension.
I found several examples for PowerShell but no examples for Xplat cli.
I would like, for example, to run the following HelloWorld PowerShell script:
New-Item -ItemType directory -Path C:\HelloWorld
After reading documentation I should be able to run a CustomExtensionScript by executing something like this (the following command does not work):
azure vm extension set xplattest3 CustomScriptExtension Microsoft.Compute 1.4 -i '{"URI":["https://gist.githubusercontent.com/tk421/8b7dd37145eaa8f82e2f/raw/36c11aafd3f5d6b4af97aab9ef5303d80e8ab29b/azureCustomScriptExtensionTest"] }'
I think that the problem is the parameter -i. I have not been able to find an example on Internet. There are some references and documentation such as MSDN and Github, but no examples.
Therefore, my question: How to execute a PowerShell script after creating a Windows VM in Azure using the xplat cli ?
Please note that the my current approach is a CustomScriptExtension, but anything that allows to bootstrap a configuration script will be considered!
EDIT How do I know it is failing ?
After I run the command azure vm extension ...:
xplat cli confirms that the command has been executed properly.
As per MSDN documentation, the folder C:\Packages\Plugins\Microsoft.Compute.CustomScriptExtension\ is created, but there is no script downloaded to C:\Packages\Plugins\Microsoft.Compute.CustomScriptExtension\{version-number}\Downloads\{iteration}
The folder C:\HelloWorld is not created, which means that the contents of the script has not been executed.
I cannot find any sort of logs or a trace to know what happened. Does anyone knows where can I find this information ?
The parameters (The Json) that I used after reading the MSDN documentation were not correct. However, you can get clues of the correct parameters by reading the C# code.
And the final command is:
azure vm extension set xplattest3 CustomScriptExtension Microsoft.Compute 1.4 -i '{"fileUris":["https://macstoragetest.blob.core.windows.net/testcontainername/createFolder.ps1"], "commandToExecute": "powershell -ExecutionPolicy Unrestricted -file createFolder.ps1" }'
This command successfully creates the C:\HelloWorld directory.
NOTE: I decided to upload the script to Azure as I read in a post and in the documentation that is mandatory. However I just made a test to download the original script from Github and it is working fine, so I guess that the documentation is a bit outdated.
EDIT: I created an detailed article that explains how to provision windows servers with xplat-cli in Azure.