14 Page Monitors


Your goal today is to become more familiar with the tools for monitoring pages, creating feeds, and manipulating feeds.

Page monitors

  • Look at Page Monitoring Software Examples (from the lecture).
  • Use WatchThatPage to create page monitors that will be emailed to your account.
    • There's no limit on the number of pages that you can monitor with this Web site.
    • On this page, read “What is a channel?” This explains how to use channels to enable keyword matching for your page monitors. If you don't need keyword matching, then you need not worry about this.
  • I would be surprised if you end up defining fewer than a dozen page monitors by the end of the semester — or fewer than 3 by the end of today's class.

Using other people's work

  1. Search through the Dapps at Dapper.net and see if there are any tools that you can use for your own project. If so, add it to your blogroll (modifying it if necessary).
  2. Search through the popular Pipes at Yahoo Pipes and, again, see if there are any tools that you can use for your own project.
    • If so, add it to your blogroll (modifying it if necessary).
    • Just as importantly, add a link and a description to this page for useful RSS filters. It would surprise me if each of you don't add one or two filters/tools to this page.
  3. Go back to some “shotgun-style” RSS feeds that you have come across before and have been afraid to add to your blogroll simply because of the volume of information you knew you would get.
    • Create a Yahoo Pipe for that feed that focuses the feed onto one topic. You could possibly spend a whole semester playing with Yahoo Pipes and learning all of its ins-and-outs. You can search for the pipe called “companyinfo” (I created it) to see an example of a simple pipe. I highlighted lots of useful documentation on the page monitors page.
    • Add these feeds to your blogroll.
    • Also, if you create a pipe that other people might find useful, please add it to this page for useful RSS filters. You should also consider writing a blog entry about it.

Feed creation

The following are more complex and are not always needed, but are good tools to have in your toolbox. Expect to struggle a bit as you go through these exercises.

  1. There's two primary feed creation Web sites that we're going to use.
    • Feed43 is appropriate for creating feeds from pages with a list of information or just a piece of information that you want to capture in a feed.
    • Dapper is appropriate for creating feeds from pages that are created as the result of entering data into a search box. It's also a really rich source of previously created tools that you might want to take advantage of.
  2. We're going to look at Dapper first.
    • Go to the site and search for “finance”; see what Dapps have already been created. I found one called “Google Finance News” that looks pretty useful. You might find others (actually a lot of others). Let's look at bit at the Dapp for Google Finance News :
      1. We want to create an RSS feed, so set “Choose format” to “RSS Feed”.
      2. Be sure to click on the “Go” button after choosing “RSS Feed”.
      3. You now need to define the parts of the RSS feed.
        1. The title should be “News A”.
        2. The item text should be News C, D, & B.
      4. Enter the Stock Ticker in the box and click on “Update Input”.
      5. Click on “Get a nice short URL”.
      6. You do not want users to be able to supply the input values (i.e., the stock ticker). You want to define this RSS to deliver information about this particular stock.
      7. Give the service (Dapp; or RSS feed, in this case) a name (e.g., MSFTgooglefinance) or something like that.
      8. Click on “Save Service”.
      9. You can now add this RSS feed to your feed reader.
    • You can create other RSS feeds for other stocks based on this same Google Finance Dapp.
    • Search for other Dapps and see if any might be useful for either one of your projects. You might also go to that same page that you went to above and explore the tags identifying already existing Dapps.
  3. Now we're going to take a look at Feed43
    1. Let's make a feed based on the News page at Ross .
      1. Be sure to create an account and get logged in.
      2. Go to the “My Feeds” page.
      3. Click on the link to Create a new feed
      4. Enter the address http://www.bus.umich.edu/NewsRoom/ into the “Address” box and click on the “Reload” button.
      5. Wait a few seconds and the HTML for the page should appear in a new text box on the page.
      6. Look at the text of the page itself. What's the first news item? When I'm looking at it, it is “The People Paradox blah blah”.
      7. Go back to the page with the HTML. Search for the text of the title in the HTML.
      8. Now find all of the text for a row in the HTML table. Again, when I'm looking at it, this text is like the text on this page.
      9. In the box titled “Global Search Pattern” under “Define extraction rules”, put the following text: Recent News{%}
        • This code can be interpreted as find all matches for news after the string Recent News''.
      10. Copy all of this data (for the row above) to the text box below “Item (repeatable) Search Pattern”.
      11. Now, you want the text in the box to end like the text in this search pattern.
        • This means start with <tr but throw away everything up to the anchor tag. Now, in the anchor tag, save the HREF value and the text of the title. Next, throw away any text that is after the anchor tag.
        • So, at the end of this, we are left with just two values from each news item.
      12. Click on the “Extract” button.
      13. There should now be something like 20 news items in the text box below. If not, then you need to revise the search pattern.
      14. In the text box “Clipped data”, there should be a series of {%1} and {%2}
      15. Scroll down the page to the RSS feed properties.
      16. Clean up the feed title and feed description.
      17. Scroll down the page to the RSS item properties.
      18. The Item Title Template should be {%2}
      19. The Item Link Template should be {%1}
      20. Click on the Preview button.
      21. Once the RSS feed looks like you want it to, click on “Change file name” in order to make the feed name something more readable. For this example, make it “youruniqnameUMrossnews” (since everyone else is making a feed on this same page). Normally, you would just name it something like “michiganrossnews”.
      22. You can now add this feed to your feed reader. You have created a feed for a page that doesn't have a feed. When the page changes, you will be notified in the RSS feed.
  4. While looking on the Web and gathering pages for your projects, be on the look-out for pages like this (i.e., that have information that you're interested in, that don't have RSS feeds, but that you wish had RSS feeds). If you find them, use either Dapper or Feed43 to create feeds.
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License