How to close / cancel a pop-up modal / dialog window in Applescript - macos

Question

I'm trying to fix a particular problem in this Applescript I made: https://gist.github.com/jwmann/08daed8a905cfbf4ff96
Context:
It's an Applescript where you select a song in a VLC playlist, run the Applescript and it will trash the song from the original location.
The Problem:
The problem occurs when a user attempts to delete a currently playing, single song where the VLC playlist is the size of just the single song.
Now VLC will explode if you try to delete a song that it's currently playing. To workaround this I've made the script stop VLC from playing and trying again to delete it.
Now if this workaround happens, VLC will no longer be playing. In a multi-song playlist, this can't be annoying. So the script will continue playing VLC at the end of the script. However, after deleting the song from a single song playlist, it will remove the song from the playlist, leaving no songs and therefore no playlist.
So when the script tries to play, it'll open a new window / dialog / modal to allow the user to find something to play. This is something I don't want to happen.
What I'm trying to do:
I need a way to:
Detect the correct Window
Tell that window to Close
Information I've gathered:
This is the window I'm trying to Cancel
This is the data that Accessibility Inspector shows me about the window.
<AXApplication: “VLC”>
<AXWindow: “Open Source”>
Attributes:
AXFocused: “0”
AXFullScreen: “0”
AXTitle: “Open Source”
AXPosition (W): “x=993 y=276”
AXGrowArea: “(null)”
AXMinimizeButton: “(null)”
AXDocument: “(null)”
AXSections (W): “<array of size 1>”
AXCloseButton: “(null)”
AXMain: “0”
AXFullScreenButton: “(null)”
AXProxy: “(null)”
AXDefaultButton: “<AXButton: “Open”>”
AXMinimized: “0”
AXChildren: “<array of size 8>”
AXRole: “AXWindow”
AXParent: “<AXApplication: “VLC”>”
AXTitleUIElement: “<AXStaticText>”
AXCancelButton: “<AXButton: “Cancel”>”
AXModal: “1”
AXSubrole: “AXDialog”
AXZoomButton: “(null)”
AXRoleDescription: “dialog”
AXSize: “w=574 h=402”
AXToolbarButton: “(null)”
AXFrame: “x=993 y=276 w=574 h=402”
AXIdentifier: “_NS:40”
Actions:
AXRaise - raise
Things I've tried:
tell application "System Events"
close window "Open Source" of application "VLC"
end tell
tell application "System Events"
click button "Cancel" of window "Open Source" of application "VLC"
end tell
tell application "System Events"
cancel window "Open Source" of application "VLC"
end tell
tell application "System Events" to tell (window 1 of application "VLC" whose subrole is "AXDialog") to close
In every example, System Events or VLC (I tried both) can't seem to find the window "Open Source" even though it's clearly called "Open Source" in the inspector. It's not a sheet, it's a window. I don't understand why I can't find this window.

Answer 1

Thanks to #l'L'l for recommending me Script Debugger 5, it allowed to view the data of the windows in a readable way and even provide code on calling specific elements.
How I found out was I ran this code:
tell application "VLC" to activate
tell application "System Events"
tell process "VLC"
set myUI to every UI element
end tell
end tell
The reason being is that even though I knew the window's name, Applescript couldn't ever find it, so I needed a way to see everything.
Within that, I found this:
If you look at the metadata of this window, you'll see that it's SUPER broken.
No wonder Applescript could never find it, it's barely a window.
From that list, it provides the exact code to reference parts of the window, including, the "Cancel" button.
This is the code that cancels my window:
tell application "System Events"
tell its application process "VLC"
tell its window "Open Source"
click button "Cancel"
end tell
end tell
end tell
So searching every UI Element was definitely useful and Script Debugger 5 definitely helped.