I am trying to download a track from Soundcloud using the ruby sdk (soundcloud 0.2.0 gem) with an app. I have registered the app on soundcloud and the client_secret is correct. I know this because I can see my profile info and tracks using the app.
Now when I try to download a track using the following code
#track = current_user.soundcloud_client.get(params[:track_uri])
data = current_user.soundcloud_client.get(#track.download_url)
File.open("something.mp3","wb"){|f|f.write(data)}
and when I open the file it has nothing in it. I've tried many approaches including the following one,
data = current_user.soundcloud_client.get(#track.download_url)
file = File.read(data)
And this one gives me an error
can't convert nil into String
on line 13 which is in
app/controllers/store_controller.rb:13:in `read'
that is the File.read function.
I have double checked that the track I am trying to download is public and downloadable.
I tried to test the download_url that is being used explicitly by copying it from console and sending a request using Postman and it worked. I am not sure why it is not working with the app when other things are working so well.
What I want to do is to successfully be able to either download or at least get the data which I could store somewhere.
Version details : -
ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-linux]
Rails 3.2.18
soundcloud 0.2.0
There are few assumptions that you have to understand before doing this thing.
Not every track on SoundClound can be downloaded! Only tracks that are flagged as downloadable can be downloaded - your code has to consider that option!
Your track URL has to be "resolved" before you get to download_url and after you get download_url you have to use your client_id to get the final download URL.
Tracks can be big, and downlowding them requires time! You should never do tasks like this straight from your Rails app in your controller or model. If the tasks runs longer you always use some background worker or some other kind of background processing "thing" - Sidekiq for example.
Command-line client example
This is example of working client, that you can use to download tracks from SoundClound. Its using official Official SoundCloud API Wrapper for Ruby, assumes that you are using Ruby 1.9.x and its not dependent on Rails in any way.
# We use Bundler to manage our dependencies
require 'bundler/setup'
# We store SC_CLIENT_ID and SC_CLIENT_SECRET in .env
# and dotenv gem loads that for us
require 'dotenv'; Dotenv.load
require 'soundcloud'
require 'open-uri'
# Ruby 1.9.x has a problem with following redirects so we use this
# "monkey-patch" gem to fix that. Not needed in Ruby >= 2.x
require 'open_uri_redirections'
# First there is the authentication part.
client = SoundCloud.new(
client_id: ENV.fetch("SC_CLIENT_ID"),
client_secret: ENV.fetch("SC_CLIENT_SECRET")
)
# Track URL, publicly visible...
track_url = "http://soundcloud.com/forss/flickermood"
# We call SoundCloud API to resolve track url
track = client.get('/resolve', url: track_url)
# If track is not downloadable, abort the process
unless track["downloadable"]
puts "You can't download this track!"
exit 1
end
# We take track id, and we use that to name our local file
track_id = track.id
track_filename = "%s.aif" % track_id.to_s
download_url = "%s?client_id=%s" % [track.download_url, ENV.fetch("SC_CLIENT_ID")]
File.open(track_filename, "wb") do |saved_file|
open(download_url, allow_redirections: :all) do |read_file|
saved_file.write(read_file.read)
end
end
puts "Your track was saved to: #{track_filename}"
Also note that files are in AIFF (Audio Interchange File Format). To convert them to mp3 you do something like this with ffmpeg.
ffmpeg -i 293.aif final-293.mp3
Related
For example, I tried to find a manual for OAuth package here but I get Ooops page missing here. I tried rim utility as instructed in Manual pages for Ruby functions from command line but version not documented
= OAuth
(from gem oauth-0.5.3)
------------------------------------------------------------------------------
------------------------------------------------------------------------------
= Constants:
OUT_OF_BAND:
request tokens are passed between the consumer and the provider out of band
(i.e. callbacks cannot be used), per section 6.1.1
PARAMETERS:
required parameters, per sections 6.1.1, 6.3.1, and 7
RESERVED_CHARACTERS:
reserved character regexp, per section 5.1
VERSION:
[not documented]
Because of missing manuals, I have tried to find tools to list methods of packages like in IDE like Dropdown list for ruby methods of a package in RubyMine IDE? and Vim plugin to autocomplete method names of Ruby packages. I haven't yet found a solution there and, here, I want to keep the focus on the manuals and proper ways to find them in Ruby.
Is there some central place for Ruby package manuals like R's cran? And if not, how do authors of public Ruby packages usually document their plugins? Is the easiest way to see which methods each package has to read the source?
This is not the easiest way but you can unpack any gem file corresponding to a package where you may be able to find demonstrations about the usage of each package.
Example about OAuth
You can download the gem file and unpack it to read the source in the following way
wget https://rubygems.org/downloads/oauth-0.5.3.gem
gem unpack oauth-0.5.3.gem
where you find the README with a demonstration about the usage.
README.rdoc
== Demonstration of usage
We need to specify the oauth_callback url explicitly, otherwise it defaults to "oob" (Out of Band)
#callback_url = "http://127.0.0.1:3000/oauth/callback"
Create a new consumer instance by passing it a configuration hash:
#consumer = OAuth::Consumer.new("key","secret", :site => "https://agree2")
Start the process by requesting a token
#request_token = #consumer.get_request_token(:oauth_callback => #callback_url)
session[:token] = request_token.token
session[:token_secret] = request_token.secret
redirect_to #request_token.authorize_url(:oauth_callback => #callback_url)
When user returns create an access_token
hash = { oauth_token: session[:token], oauth_token_secret: session[:token_secret]}
request_token = OAuth::RequestToken.from_hash(#consumer, hash)
#access_token = #request_token.get_access_token
#photos = #access_token.get('/photos.xml')
Now that you have an access token, you can use Typhoeus to interact with the OAuth provider if you choose.
require 'oauth/request_proxy/typhoeus_request'
oauth_params = {:consumer => oauth_consumer, :token => access_token}
hydra = Typhoeus::Hydra.new
req = Typhoeus::Request.new(uri, options) # :method needs to be specified in options
oauth_helper = OAuth::Client::Helper.new(req, oauth_params.merge(:request_uri => uri))
req.options[:headers].merge!({"Authorization" => oauth_helper.header}) # Signs the request
hydra.queue(req)
hydra.run
#response = req.response
All functions in Ruby OAuth find . -iname "*.rb"
./lib/oauth/cli/authorize_command.rb
./lib/oauth/cli/base_command.rb
./lib/oauth/cli/help_command.rb
./lib/oauth/cli/query_command.rb
./lib/oauth/cli/sign_command.rb
./lib/oauth/cli/version_command.rb
./lib/oauth/cli.rb
./lib/oauth/client/action_controller_request.rb
./lib/oauth/client/em_http.rb
./lib/oauth/client/helper.rb
./lib/oauth/client/net_http.rb
./lib/oauth/client.rb
./lib/oauth/consumer.rb
./lib/oauth/core_ext.rb
./lib/oauth/errors/error.rb
./lib/oauth/errors/problem.rb
./lib/oauth/errors/unauthorized.rb
./lib/oauth/errors.rb
./lib/oauth/helper.rb
./lib/oauth/oauth.rb
./lib/oauth/oauth_test_helper.rb
./lib/oauth/request_proxy/action_controller_request.rb
./lib/oauth/request_proxy/base.rb
./lib/oauth/request_proxy/curb_request.rb
./lib/oauth/request_proxy/em_http_request.rb
./lib/oauth/request_proxy/jabber_request.rb
./lib/oauth/request_proxy/mock_request.rb
./lib/oauth/request_proxy/net_http.rb
./lib/oauth/request_proxy/rack_request.rb
./lib/oauth/request_proxy/rest_client_request.rb
./lib/oauth/request_proxy/typhoeus_request.rb
./lib/oauth/request_proxy.rb
./lib/oauth/server.rb
./lib/oauth/signature/base.rb
./lib/oauth/signature/hmac/sha1.rb
./lib/oauth/signature/plaintext.rb
./lib/oauth/signature/rsa/sha1.rb
./lib/oauth/signature.rb
./lib/oauth/token.rb
./lib/oauth/tokens/access_token.rb
./lib/oauth/tokens/consumer_token.rb
./lib/oauth/tokens/request_token.rb
./lib/oauth/tokens/server_token.rb
./lib/oauth/tokens/token.rb
./lib/oauth/version.rb
./lib/oauth.rb
I'm getting the above error in a gem with this code snippet
Savon.configure do |config|
config.log = false
config.log_level = :error
HTTPI.log = false
end
This code used to pass in past runs on Travis, so I'm not sure why this changed when I altered the Readme.
Part of this confusion comes from my situation--inheriting a gem to maintain--along with this line in the gemspec:
gem.add_dependency 'savon'
There's no version number specified, so the newest run switched over to using Savon 2, which ditched the Savon.configure global behavior. If you're in the same boat as me, changing this line to the last pre-2.0 version of Savon will resolve the issue:
gem.add_dependency 'savon', '~>1.2.0'
Then bundle install and you should be good.
Or you want to upgrade your code. I know I do.
Savon.configure was removed from Savon 2.0 because the "problem was global state". The quickest way to keep the behavior the same in your app would be to define a app-level global hash in the same place. You'd then pass this hash into every Savon.client call you make. For instance:
# Where Savon.configure was called
APP_OPTS = {
# disable request logging, silences HTTPI as well
log: false,
# Don't log Laundry xmls to STDOUT
log_level: :error,
#... etc
}
# Elsewhere
#client = Savon::Client.new(APP_OPTS)
I'd consider this a starting point to migrating to the 2.0 configuration style. Ideally, you should always consider the client-specific 2.0 options available when initializing each Savon client.
I am attempting to create a minimal Websocket implementation using Cramp framework.
Where as Cramp successfully renders normal web content, I run into trouble when I try to use HTML5 websockets.
My action class is as follows :
Cramp::Websocket.backend = :thin
class HomeAction < Cramp::Action
self.transport = :websocket
keep_connection_alive
on_data :recv_data
def recv_data data
puts "got message"
puts "#{data}"
render "Hello world"
end
end
My javascript code is as follows :
$(function(){
window.socket = new WebSocket("ws://localhost:3000/game");
socket.onmessage = function(evt){
console.log(evt.data);
socket.close();
}
socket.onclose = function(evt) {
console.log("end");
}
socket.onopen = function() {
console.log("Now open!");
socket.send("Hello");
}
})
The server (thin) detects when data is sent but the text that is read is garbled.
the encoding of the data is ASCII-8BIT (puts data.encoding prints "ASCII-8BIT"). However forcing UTF encoding through data.force_encoding('UTF-8') does not resolve the issue. In addition after forcing encoding - data.valid_encoding? returns false where as it was true before forcing.
I have tested the app in ruby-1.8.7 as well as ruby-1.9.3 . The output is same in both scenarios.
Another weird thing is that in client side the onmessage event is never fired.
Also, if I remove keep_connection_alive call from HomeAction the connection immediately terminates after the data is received and still the client does not receive the data being sent by server ("Hello world").
I have tested the app in Google chrome (latest version) and Mozilla firefox (latest version). The problem remains exactly the same in both of them. My operating system is Ubuntu 12.04 LTS (Precise Pangolin).
Any help in this regard would be strongly appreciated.
I have been running into the same thing, and it seems to be an issue with the released version of the cramp 0.15.1 gem versus what you get from the github repo (https://github.com/lifo/cramp) thought is still marked as 0.15.1.
Try this experiment which works for me:
Clone the GH repo locally
Copy in the bin/ and lib/ folders, as well as the cramp.gemspec file from the repo to your test cramp project
Change your gemfile, instead of just
gem 'cramp'
Include the local copy of code:
gemspec
gem 'cramp', :path => File.dirname(__FILE__)
Erase your Gemfile.lock and re-bundle, see that bundler now reports it will use the local copy of the cramp gem
Try your app again, in my scenario, this now works exactly as expected.
It would appear there is either a fix in github they have not released yet (but have not incremented the working version in their gemspec) or some other version snafu, but either way the code in GH works whereas a "gem install cramp" doesn't give you working code for websockets.
So here's what I'm attempting to do. I'm building an ember.js application, with a java backend running on GAE.
I'm using handlebars, but I want them divided up into separate files, not just all pasted into the index.html.
Via the ember.js irc I was turned on to rake-pipeline along with minispade
Along with the web filters and a custom handlebars filter I started building the assetfile. I don't know Ruby, or gem files, etc.
So I'm trying to figure out the best way to be able to compile my coffeescript/handlebars files on the fly, minispade them, but keep the individual files accessible while in dev mode so I can debug them. What makes that hard is that the rake pipeline is running on a different port than GAE. So I'm not sure exactly how to handle this. Do I make my index file in GAE point to individual files at the 9292 port (rakep) during development, but in production mode point to the fully concatenated version? I'm not sure.
So I was attempting to do that here: https://gist.github.com/1495740 by having only one section that was triggered by the 'build' flag. Not even sure if that works that way.
I know there's a lot of confusion here. Apologies, like I said I'm not even remotely familiar with the Ruby style of doing things.
Since you're not a Ruby person, here are the most reliable steps for getting a stock OSX environment set up with rake pipeline:
Step 1: Install bundler
# on OSX, using built-in Ruby
$ sudo gem install bundler --pre
Step 2: Create a Gemfile
# inside your app directory
$ bundle init
# will create a file named Gemfile in the root
Step 3: Add rake-pipeline to the Gemfile
# inside the Gemfile
gem "rake-pipeline-web-filters"
Step 4: Install your gems
$ bundle install --binstubs
Step 5: Set up Assetfile
However you were already doing it...
Step 6: Run Rake::Pipeline
# to run the preview server
$ bin/rakep
# to build your assets
$ bin/rakep build
Rake::Pipeline.build is the method that evaluates an Assetfile. You can imagine that your entire Assetfile is wrapped inside a Rake::Pipeline.build {} block; you shouldn't ever need to write one inside an Assetfile.
Some of the filters in the docs are hypothetical, most of those docs were written before there were any filters at all. A CoffeeScript compiler has been recently added, though.
As to your main question, I'm not sure there's a clean way to do it with the current rakep implementation. An Assetfile is just Ruby, though, so it's possible to hack something together that should work. Here's how I would write yours:
require "json"
require "rake-pipeline-web-filters"
require "rake-pipeline-web-filters/helpers"
class HandlebarsFilter < Rake::Pipeline::Filter
def initialize(&block)
block ||= proc { |input| input.sub(/\.handlebars$/, '.js') }
super(&block)
end
def generate_output(inputs, output)
inputs.each do |input|
output.write "return Ember.Handlebars.compile(#{input.read.to_json})"
end
end
end
# process all js, css and html files in app/assets
input "assets"
# processed files should be outputted to public
output "public"
# process all coffee files
match "**/*.coffee" do
# compile all CoffeeScript files. the output file
# for the compilation should be the input name
# with the .coffee extension replaced with .js
coffee_script
# The coffee_script helper is exactly equivalent to:
# filter Rake::Pipeline::Web::Filters::CoffeeScriptCompiler
end
match "**/*.js" do
minispade
if ENV['RAKEP_ENV'] == "production"
concat "application.js"
else
concat
end
end
match "**/*.handlebars" do
filter HandlebarsFilter
minispade
concat "templates.js"
end
The if ENV['RAKEP_ENV'] bit reads an environment variable to decide whether to concatenate your JS to a single file.
So now you can run RAKEP_ENV="production" rakep build for a concatenated build, or just rakep build for a development build.
Is there anyway to get Ruby's require statement to download a file from somewhere like github rather than just the local file system?
Update: Sorry I should have made the question clearer. I want to download a file that contains Ruby module and import it into my script rather than just downloading an image or some other arbitrary file within my script.
In other words something like this
require 'http:\\github.com\myrepo\snippet.rb'
puts 'hi'
By default, this is not possible. Also, it's not a good idea for security reasons.
You have a couple of alternatives. If the file you want to include is a Gem and Git repository, then you can use Bundler to download and package the dependency in your project. Then you'll be able to require the file directly in your source code.
This is the best and safest way to include an external dependency.
If you trust the source and you really know what you are doing, you can download the file using Net::HTTP (or any other HTTP library) and eval the body directly in your Ruby code.
You can package everything in a custom require_remote function.
You could download and eval it
require "open-uri"
alias :require_old :require
def require(path)
return false if $".include?(path)
unless path=~ /\Ahttp:\/\/
return require_old(path)
end
eval(open(path).read)
$"<< path
true
end
Be aware, this code has no error checking for network outages nonexisting files, ... . I also believe it is in general not a good idea to require libraries this way, there are security and reliability problems in this approach. But maybe you have a valid usecase for this.
you can include a remote gem from within Gemfiles then it will download when you run bundle install
After reading this question and answers I wanted something a little more bullet proof and verbose that used a paradigm of creating a local file from a repo and then requiring it, only if it didn't already exist locally already. The request for the repo version is explicit via the method repo_require. Used on files you control, this approach improves security IMO.
# try local load
def local_require(filename, relative_path)
relative_flname = File.join(relative_path, filename)
require_relative(relative_flname)
end
# try loading locally first, try repo version on load error
# caution: only use with files you control access to!
def repo_require(raw_repo_prefix, filename, relative_path = '')
local_require(filename, relative_path)
rescue LoadError => e
puts e.message
require 'open-uri'
tempdir = Dir.mktmpdir("repo_require-")
temp_flname = File.join(tempdir, File.basename(filename))
return false if $LOADED_FEATURES.include?(temp_flname)
remote_flname = File.join(raw_repo_prefix, filename)
puts "file not found locally, checking repo: #{remote_flname}"
begin
File.open(temp_flname, 'w') do |f|
f.write(URI.parse(remote_flname).read)
end
rescue OpenURI::HTTPError => e
raise "Error: Can't load #{filename} from repo: #{e.message} - #{remote_flname}"
end
require(temp_flname)
FileUtils.remove_entry(tempdir)
end
Then you could call repo_require like this:
repo_require('https://raw.githubusercontent.com/username/reponame/branch',
'filename', 'relative_path')
The relative_path would the the relative path you would use for the file if the repo was locally installed. For example, you may have something like require_relative '../lib/utils.rb'. In this example filename='lib/utils.rb' and relative_path='..'. This information allows the repo url to be constructed correctly as it does not use the relative path portion.