Applescript: Truncated feed URL - applescript

Question

I'm running into errors with this simple script. It's a validator to validate RSS feed with multiple validation sites using Safari. Everything works fine as long as the feed does not contain special characters or anything after the = sign.
The script should validate the feed that was copied to the clipboard.
For example, this feed works fine: http://thefirst.libsyn.com/rss
This feed gets truncated after ?id: https://www.npr.org/rss/podcast.php?id=510298
This is only happening on the Podbase validator site.
If I could get the script to click the Validate and Go buttons, that would be amazing, but this is pretty basic…just stuck as to why the feed is getting truncated.
set feed_url to the clipboard as string
set the podbaseurl to "http://podba.se/validate/?url=" & feed_url
set the feedvalidatorurl to "http://feedvalidator.org/check.cgi?url=" & feed_url
set the castfeedurl to "http://castfeedvalidator.com/?url=" & feed_url
tell application "Safari"
make new document
open location podbaseurl
open location feedvalidatorurl
open location castfeedurl
end tell

Answer 1

The problem is that https://podba.se/validate only makes it look like a single GET request is enough, whereas clicking the Go button interactively performs many individual GET requests behind the scenes whose results are pieced together on the current page (and the URL is then modified to include the submitted feed URL).
In other words: even solving the (odd) truncation problem wouldn't be enough.
Therefore, your best bet is indeed to simulate an interactive submission of a feed URL, which requires filling an input box with the feed URL and pressing the submission button.
Interactive submission must be simulated for the http://castfeedvalidator.com site as well, where pressing the submission button is sufficient, however.
(As you report, even though inspecting the request sent by the submission button shows that a variation of your URL - which only prepares the feed URL for submission - can be used to instantly submit it, doing so doesn't render the results correctly (missing styles)).
The following code implements both suggestions (the simulated-interaction approach was adapted from this answer of mine):
# Sample value
set the clipboard to "https://www.npr.org/rss/podcast.php?id=510298"
set feed_url to the clipboard as string
# Base URL only; the feed URL will be submitted by simulated interaction below.
set the podbaseurl to "http://podba.se/validate/"
set the feedvalidatorurl to "http://feedvalidator.org/check.cgi?url=" & feed_url
set the castfeedurl to "http://castfeedvalidator.com/?url=" & feed_url
tell application "Safari"
activate
set newDoc to make new document
set newWin to front window
# Simulate interactive submission of feed_url at podbaseurl.
set URL of newDoc to podbaseurl
my submitUrl(newDoc, podbaseurl, feed_url)
# The feedvalidateorurl can be opened normally.
open location feedvalidatorurl
# Simulate interactive submission of feed_url at castfeedurl.
set newTab to make new tab in newWin
set URL of newTab to castfeedurl
my submitUrl(newTab, castfeedurl, feed_url)
end tell
on submitUrl(doc, target_url, feed_url)
# Synthesize the JavaScript command.
set jsCode to "
(function () { // Use a closure to avoid polluting the global namespace.
function doIt(t) { // Helper function
if (doIt.done) return; // Already successfully called? Ignore.
try {
// Perform the desired action - may fail if invoked too early.
if (/^http:\\/\\/podba.se/.test('" & target_url & "')) {
document.querySelector('#url-input').value = '" & feed_url & "';
document.querySelector('#url-input').dispatchEvent(new Event('input'));
setTimeout(function() { document.querySelector('#go-button').click(); }, 0);
} else { // http://feedvalidator.org
document.querySelector('.btn-subscribe').click()
}
} catch(e) {
return; // Return without setting the success 'flag'.
}
doIt.done=true; // Set success 'flag' as a property on this function object.
};
// Attach a listener to the window's load event for invoking the helper function then.
window.addEventListener('load', doIt);
// If the document signals readiness -- which may still be too early, we can't know --
// also try to invoke the helper function *directly*.
if (document.readyState === 'complete') { doIt(); }
})();
"
# Execute the JavaScript command in the target page.
tell application "Safari"
tell doc to do JavaScript jsCode
end tell
end submitUrl