Spring Integration RSS Feed - 403 Error - spring

Question

I am trying to use Spring Integration to poll RSS feeds and I'm using SO user feed URL as an example. However when the application runs I get the following error:
Disconnected from the target VM, address: '127.0.0.1:58603', transport: 'socket'
2016-03-04 19:34:36.252 ERROR 2345 --- [ask-scheduler-4] o.s.integration.handler.LoggingHandler : org.springframework.messaging.MessagingException: Failed to retrieve feed at url 'http://stackoverflow.com/feeds/user/813852'; nested exception is com.rometools.fetcher.FetcherException: Authentication required for that resource. HTTP Response code was:403
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.getFeed(FeedEntryMessageSource.java:216)
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.populateEntryList(FeedEntryMessageSource.java:182)
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.doReceive(FeedEntryMessageSource.java:157)
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.receive(FeedEntryMessageSource.java:122)
at org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:175)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:224)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.access$000(AbstractPollingEndpoint.java:57)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:176)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:173)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:330)
at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:55)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:51)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:324)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.rometools.fetcher.FetcherException: Authentication required for that resource. HTTP Response code was:403
at com.rometools.fetcher.impl.AbstractFeedFetcher.throwAuthenticationError(AbstractFeedFetcher.java:183)
at com.rometools.fetcher.impl.AbstractFeedFetcher.handleErrorCodes(AbstractFeedFetcher.java:170)
at com.rometools.fetcher.impl.HttpURLFeedFetcher.retrieveAndCacheFeed(HttpURLFeedFetcher.java:186)
at com.rometools.fetcher.impl.HttpURLFeedFetcher.retrieveFeed(HttpURLFeedFetcher.java:140)
at com.rometools.fetcher.impl.HttpURLFeedFetcher.retrieveFeed(HttpURLFeedFetcher.java:99)
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.getFeed(FeedEntryMessageSource.java:204)
... 22 more
The URL loads fine through the browser (even in incognito mode so the error message is probably misleading). I downloaded an RSS reader and the feed worked fine.
This is how I configured the feed:
<int-feed:inbound-channel-adapter id="stackOverflow"
channel="log"
url="http://stackoverflow.com/feeds/user/813852">
<int:poller fixed-rate="10000" max-messages-per-poll="100" />
</int-feed:inbound-channel-adapter>
<int:logging-channel-adapter id="log" level="DEBUG" />
I also tried another RSS feed URL and it worked. Any ideas what could be wrong?

Answer 1

I just ran a wireshark trace and it looks like StackOverflow doesn't like the java user agent:
User-Agent: Java/1.8.0_66\r\n
...
<p>The owner of this website (stackoverflow.com) has banned your access based
on your browser's signature (27e797571e1923de-ua21).</p>\n
Setting a custom user agent goes around this restriction:
public class CustomFeedFetcher extends HttpURLFeedFetcher {
#Override
public void setUserAgent(String s) {
super.setUserAgent("AgentName");
}
}