AppleScript cURL and parse URL - applescript

Question

I am working on a Automator workflow, I am passing list of URLs to the "Run Applescript" and I need to fetch the contents of on each page, concatenate and pass it to a BBedit (or any other text editor).
on run {input, parameters}
tell application "BBEdit"
activate
set astid to AppleScript's text item delimiters
set startHere to "<tbody>"
set stopHere to "</tbody>"
repeat with anItem in input
set blurb0 to (do shell script "curl " & anItem)
set AppleScript's text item delimiters to startHere
set blurb1 to text item 2 of blurb0
set AppleScript's text item delimiters to stopHere
set blurb2 to text item 1 of blurb1
set AppleScript's text item delimiters to astid
return blurb2
beep
end repeat
end tell
end run
The current code only properly gets only the contents from first URL. Can anybody fix this?

Answer 1

This subroutine may be what you need (if you're using Safari)...
on getSource(this_URL)
tell application "Safari"
activate
set the URL of the current tab of document 1 to this_URL
set the |source| to the source of the front document
end tell
tell application "TextEdit"
activate
set the text of the front document to the source
end tell
quit application "Safari"
end getSource
Call it using:
repeat with anItem in input
getSource(input)
end repeat
I hope this helps!

Answer 2

Because you are inside the repeat with anItem in input loop it does all your work for the first item and the return exits the loop and in fact the whole Automator action. I think you have never heard the beep from your script ;-)
On the other side I'm wondering why you want BBEdit to do the cUrl and sort work for you. All called handlers don't belong to BBEdit...
I think your handler should look like this:
on run {input, parameters}
-- define a few parameters
set astid to AppleScript's text item delimiters
set startHere to "<tbody>"
set stopHere to "</tbody>"
-- define a list to store all found content
set allFoundContent to {}
repeat with anItem in input
set blurb0 to (do shell script "curl " & anItem)
set AppleScript's text item delimiters to startHere
set blurb1 to text item 2 of blurb0
set AppleScript's text item delimiters to stopHere
-- put the found content at the end of the list
set end of allFoundContent to text item 1 of blurb1
set AppleScript's text item delimiters to astid
end repeat
-- from here you have three possibilities:
-- 1. return the list to next Automator action (uncomment the next line):
-- return allFoundContent
-- 2. concatenate the list with a delimiter you like (here return & "------" & return)
-- and give it to your preferred text editor from this point (uncomment the next lines):
-- set AppleScript's text item delimiters to return & "------" & return
-- tell application "TextEdit"
-- make new document with properties {text: allFoundContent as text}
-- end tell
-- set AppleScript's text item delimiters to astid
-- 3. concatenate the list with a delimiter you like (here return & "------" & return)
-- and give it to the next workflow step, maybe a BBEdit action waiting for a string? (uncomment the next lines):
-- set AppleScript's text item delimiters to return & "------" & return
-- set returnString to allFoundContent as text
-- set AppleScript's text item delimiters to astid
-- return returnString
-- Next decision (for choice 1 or 2):
-- What do you want to give to next Automator action?
-- you can pass your input (the given URLs) (uncomment next line):
-- return input
-- or your result list (uncomment next line):
-- return allFoundContent
end run
Greetings, Michael / Hamburg