My university has a website where it posts announcements. I can't afford to miss these announcements, and at the same time, checking the website every day is kinda cumbersome. The website has no RSS feed.
The announcements are posted on a web page with the following as the format for the URL of an announcement :
http://example.com/news/detail/1/n
where n is the announcement ID, which is numeric.
When there is an announcement, the above web page (http://example.com/news/detail/1/180, for example), contains the announcement in the following format :
<div class="middleconten">
<h3>
Title </h3>
11 October, 2019
<p>
<a href='/some/link' target='_blank'>Click here for more details</a>
</p>
</div>
and when there is no announcement, (that is, when a user visits a web page with a n value, that doesn't correspond to an actual announcement ID, http://example.com/news/detail/1/1234567890, for example), the web page is as follows :
<div class="middleconten">
<h3>
</h3>
1 January, 1970
<p>
</p>
</div>
How do I make a RSS feed for the website capturing the <h3> value, the href attribute, and the date?
You will need to scrape the website regularly for new news items. You can use goquery for extracting the data.
The idea is simple. You need to generate the urls for the news section (fill in the value of n) starting from 1 and visit each url. If you find news (the structure exists), store the data. Add 1 to the n value to get the next ID. If the url doesn't contain news, stop and store the value of the number / ID of last successful news. The next time you can start from this ID instead of the beginning.
For example, I start from 1, I find the last successful news at ID 32. I save it somewhere. Next time I can start from 33 instead of 1.
When you have a database of the data extracted from the website, you can publish your own RSS feed from those. You can use a router like chi and gorilla feeds to create the rss feed.
Related
i try to develop a website use laravel. but i someing confuse mastering templete. there are two type of page
1. home page 2. category/menu page number 1 and number 2 type page header and footer page sameler, only 2 number page use a right sidebar. how to mastering the page stenderd way.. number 1 and number 2 page use only one #extends?
home page and menu page image https://imgur.com/aoUhcON https://imgur.com/aoUhcON
you can include views as well
<div>
#include('right-sidebar')
</div>
I recommend reading the docs first
https://laravel.com/docs/5.8/blade#including-sub-views
I've just created a custom template to be used on Mailchimp.
Some links come from custom merge tags and point to the website related to Mailchimp. The problem is that those links from merge tags are not being tracked by Mailchimp.
I found this and of course tried but it didn't work:
how can i tracking click in custom html template mailchimp?
As an example, one of my links looks like this:
<p mc:edit="product1_url">
<a class="product-button" href="http://mywebsite.com*|PRODURL1|*" target="_blank">BUY NOW</a>
</p>
The PRODURL1 merge tag is type text and contains a path to a product, so the resulting url looks like http://mywebsite.com/product/123456 and it works if a click it on the sent email.
Any experience or direction?
Thanks!!
Well, after trying and trying and trying more I've found out why it wasn't working.
First part of the links MUST include the /, the second part (contained on the merge tag) MUST NOT include the /).
So now with this configuration Mailchimp tracks the links:
<a class="product-button" href="http://mywebsite.com/*|PRODURL1|*" target="_blank">BUY NOW</a>
Of course now PRODURL1 = "products/123456". So the same url is rendered, but this time tracked by Mailchimp.
The German Supreme Court publishes an RSS feed of all its decisions. Unfortunately, the items in this RSS feed, rather than linking to the PDFs of the decision directly, link to a web page in which the PDF is contained in an iFrame.
The web pages are all structured in a parallel manner. For example, in a random RSS feed item's linked web page, the relative link in the source code will look like this:
<iframe border='0' src='document.py?Gericht=bgh&Art=en&Datum=Aktuell&nr=66132&Frame=4&.pdf' width='744px' height='100%'>Leider kann Ihr Browser keine eingebetteten Frames darstellen. Klicken Sie <a href='document.py?Gericht=bgh&Art=en&Datum=Aktuell&nr=66132&Frame=4&.pdf'>hier</a>, um das gewünschte Dokument zu erhalten.</iframe>
The links are all relative to the folder
http://juris.bundesgerichtshof.de/cgi-bin/rechtsprechung/
I want to convert this RSS feed into an RSS feed in which each item's link is a link directly to the PDF, so in my example the RSS feed item's link should become "http://juris.bundesgerichtshof.de/cgi-bin/rechtsprechung/document.py?Gericht=bgh&Art=en&Datum=Aktuell&nr=66132&Frame=4&.pdf".
My idea is to use Yahoo Pipes to loop through all the items of the RSS feed, follow the item's link, look at the source code of the web page and extract the string between <iframe border='0' src=' and the next ', stick the absolute folder path in front of the relative result, and re-assigning this to the item's link. My sad attempt at doing this is found here. Basically, I have no idea what to enter in the XPath module.
I have bad news for you. I'm afraid this won't be possible.
The solution in this kind of situation is to create two pipes:
A low-level pipe:
Receive a URL Input with values like this: http://juris.bundesgerichtshof.de/cgi-bin/rechtsprechung/document.py?Gericht=bgh&Art=en&az=IX%20ZR%2044/12&nr=66132
Use the XPath Fetch Page module to fetch the URL
Extract the iframe attribute, hopefully, and return as result
A higher level pipe:
Fetch your original URL with Fetch Feed
Loop over the feed items, in each iteration calling the low-level pipe using the URL field of the feed item and assign the result to an attribute
Construct the URL from the newly assigned attribute
And this would probably work. Except that, unfortunately, this website rejects Yahoo Pipes: it receives a "Forbidden 403" error when trying to fetch that page.
So this is cannot work directly with Yahoo Pipes. An alternative is if you can setup a proxy server, which could relay the requests so that the German website cannot know they are coming from Yahoo Pipes.
Btw, this is the same reason I cannot create custom feeds based on IMDB (the internet movie database). They refuse all requests coming from Yahoo Pipes.
Basically I have created a new view that shows only the puppies in list form and nothing else. Everything else is working fine except parts of the pagination.
So there are 40 puppies in this category. With 20 showing per page it shows 2 pages and works correctly. With 5 per page it shows 8 pages but doesn't work correct. When I hover over the first four page links it still shows the href to page 1 and the last four page links show href to page 2.
Here is the original / first page:
http://americasfavoritepuppybreeders.com/puppies/breed/labrador.html?tmpl=component&view=microsite
Clicking on Page 2 links goes here:
http://americasfavoritepuppybreeders.com/puppies/breed/labrador/page-1.html?tmpl=component
If I manually type in the URL that should be there, it still shows the puppies from the first page:
http://americasfavoritepuppybreeders.com/puppies/breed/labrador/page-2.html?view=microsite&tmpl=component
I need to keep the view=microsite & tmpl=component on the end of the URL.
Not sure exactly what code I should include for anyone to diagnose so please ask and I will post.
I've been working on this for 2 1/2 days now devoting my entire days to this... if anyone could help that would be so much appreciated! Thanks.
You can try create the custom template for pagination (pagination.php). Example you can find in /administrator/templates/bluestork/html/pagination.php, So, first step, in your temmplate you will create file /templates/{$your_template}/html/pagination.php. Second step, you can try use submitform() and <input type="hidden" name="view" value="microsite" /><input type="hidden" name="tmpl" value="component" /> or adjust link in pagination how you want.
I have created a page where there are various items on a page and people need to vote on them by clicking "recommend" (like how they have it on levi.store.com). The items are sorted based on the number of "recommends" they receive. The problem I am having is that there are 100 of these items, and when I try to display them it becomes way too slow. Is there a way to do this more effectively, this is some pseudo-code of what I have (I am using Wordpress)
$theCategory = 'the-item-category'; //every item is a post and is placed into this category
$items->query('cat='.$theCategory); //this gets all those items in that category
while($items->have_posts()) : $items->the_post();
<h1><?php the_title(); ?></h1>
<iframe src="http://www.facebook.com/plugins/like.php?href=<?php echo urlencode(get_permalink($post->ID)); ?>&layout=button_count&show_faces=false&width=450&action=recommend&font&colorscheme=light&height=21" scrolling="no" frameborder="0" style="border:none; overflow:hidden; width:140px; height:21px;" allowTransparency="true"></iframe>
I would recommend using the FBML version of the Like button. You can then display them on-demand, like TechCrunch does on story mouse over, or start loading them after page load (i.e. on DOM ready). Turn off automatic FBML parsing in your Facebook init and them use FB.XFBML.parse(DOM ID) to render each Like button.
Using iframes directly mean you are trying to load 100 web pages on page load. That's a lot, especially when browsers will only open a max of 8 connections per domain. Some open less. So with 100 Like buttons, it will take over a dozen "rounds" to load everything.
I'm assuming you are fetching and caching the number of Likes each story has on the server.