« Shine Gallery Uploader - A Talk at the Yahoo F2E Summit 08
php function to create date and time dropdown / select boxes »

Creating an Autotagger with Yahoo’s Term Extraction Service and YUI

So lets talk about tags…If you are an editor of a blog, photos, or even bookmarks these days you know all about tagging.  Incase you don’t what they are you should read up on them because you are missing a great thing on the internet (http://en.wikipedia.org/wiki/Tag_(metadata)).

It got me to thinking (scary I know) that perhaps it is a pain in the ass as a content developer to figure out what your tags are.  I mean common how do I know what people are searching for?  Isn’t there a better way?  Well perhaps there is.  Perhaps we can tie into some kind of a powerful open search api like what Yahoo has to tell us what the popular terms are in my article and to build me some tags off of it.

Hmm that seems to easy right?  Well guess what it is! Just look for yourself:

Test out the autotagger for yourself

So how the heck do you do that? Well simple just follow along:

The first thing you need to do is go out to http://developer.yahoo.com and register an appid this will allow you 5000 searches a day on their open ids’ which is more then enough for personal use.  You then need to look over Yahoo’s Term Extraction service at: http://developer.yahoo.com/search/content/V2/termExtraction.html to see what is required.

Also for fun lets put in a rich text editor because well lets be user friendly and realistic about an environment.  You can view what is required for that at: http://developer.yahoo.com/yui/editor/

So lets write some code for our Front End:

  1. <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
  2. <html>
  3. <head>
  4. <title>RTE Autotagger</title>
  5. <!-- Skin CSS file -->
  6. <link rel="stylesheet" type="text/css" href="http://yui.yahooapis.com/2.6.0/build/assets/skins/sam/skin.css">
  7. </head>
  8. <body class="yui-skin-sam">
  9. <h2>RTE Autotagger</h2>
  10. <form id="rtepost" method="post">
  11. <br>
  12. <textarea id="editor" name="testrte"></textarea>
  13. <br>
  14. Tags:
  15. <input type="text" id="mytags" size="50">
  16. <input type="button" value="Guess My Tags" id="mytagbtn">
  17. <img src="http://l.yimg.com/jn/images/20081008192436/ajax-loader2.gif" height="20" width="20" id="loading" style="display:none;">
  18. <!--<input type="submit" />-->
  19. </form>
  20. <!-- Utility Dependencies -->
  21. <script type="text/javascript" src="http://yui.yahooapis.com/2.6.0/build/yahoo-dom-event/yahoo-dom-event.js"></script>
  22. <script type="text/javascript" src="http://yui.yahooapis.com/2.6.0/build/element/element-beta-min.js"></script>
  23. <!-- Needed for Menus, Buttons and Overlays used in the Toolbar -->
  24. <script src="http://yui.yahooapis.com/2.6.0/build/container/container_core-min.js"></script>
  25. <script src="http://yui.yahooapis.com/2.6.0/build/menu/menu-min.js"></script>
  26. <script src="http://yui.yahooapis.com/2.6.0/build/button/button-min.js"></script>
  27. <!-- Source file for Rich Text Editor-->
  28. <script src="http://yui.yahooapis.com/2.6.0/build/editor/editor-min.js"></script>
  29. <!-- scouce for connection manager -->
  30. <script src="http://yui.yahooapis.com/2.6.0/build/connection/connection-min.js"></script>
  31. <script>
  32. (function() {
  33. var Dom = YAHOO.util.Dom,
  34. Event = YAHOO.util.Event,
  35. Lang = YAHOO.lang,
  36. Connect = YAHOO.util.Connect;
  37. var myEditor = new YAHOO.widget.Editor('editor', {
  38. height: '300px',
  39. width: '522px',
  40. dompath: true, //Turns on the bar at the bottom
  41. animate: true //Animates the opening, closing and moving of Editor windows
  42. });
  43. myEditor.render();
  45. var generateTags = function(o){
  46. var mydata = eval('(' + o.responseText + ')');
  47. //drop in tags from json object
  48. Dom.get('mytags').value=mydata;
  49. //repace all spaces, this is where you do could other filtering or you could drop in a dash between tags
  50. Dom.get('mytags').value=Dom.get('mytags').value.replace(/ /g,'');
  51. //turn off progress spinner
  52. Dom.get('loading').setAttribute('style','display:none;');
  53. }
  54. //add event to tags button
  55. Event.on('mytagbtn','click', function(){
  56. //turn on progress spinner
  57. Dom.get('loading').setAttribute('style','');
  58. //grab content of the rte window and strip out html
  59. myContent='myContent='+myEditor._getDoc().body.innerHTML.replace(/(<([^>]+)>)/ig,"");
  60. //make ajax call to our simple proxy
  61. myTagsCnt=Connect.asyncRequest('POST', 'make_tags_api.php', {
  62. success: generateTags,
  63. failure: function() {},
  64. scope: this
  65. }, myContent);
  66. });
  69. })();
  71. </script>
  72. </body>
  73. </html>

Now lets write our simple proxy api:

  1. <?php
  2. //might want to add some security here to make sure only you are hitting your api ;)
  3. //set your type as doctype as json
  4. header("Content-Type:application/json");
  5. //create curl function to do a simple proxy for yahoo search
  6. function getContextResource($url){
  7. $ch = curl_init();
  8. curl_setopt($ch, CURLOPT_URL, $url);
  9. curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
  10. $result = curl_exec($ch);
  11. curl_close($ch);
  12. return $result;
  13. }
  14. //pull your posted content from Post
  15. $myContent=$_POST['myContent'];
  16. //create url to curl, add in your appid, output, content to check and then urlencode and utf8encode your content
  17. $contextUrl = 'http://search.yahooapis.com/ContentAnalysisService/V1/termExtraction?appid=yourapid&output=json&context='.urlencode(utf8_encode($myContent));
  18. //create curl call from our url
  19. $feed = getContextResource($contextUrl);
  20. //convert to array so we can cut out the things we don't need
  21. $convertToArray = json_decode($feed, true);
  22. //only report back the values we need
  23. $cleanedArray = $convertToArray['ResultSet']['Result'];
  24. //return back a json encoded copy of the array we just cleaned up
  25. echo json_encode($cleanedArray);
  26. ?>

Wow after all of that what do we get?  Well if I copy a well written article from Yahoo Finance such as:
http://biz.yahoo.com/ap/081022/financial_meltdown.html

I get the following tags back:
treasurysecretaryhenrypaulson,apeconomics,henrypaulson,martincrutsinger, aggressivesteps,bushadministration,ysm,financialcrisis,recession,infinity, advertisement,economy,yahoo

Hmm some those are really not bad at all for just a simple search api huh.

This entry was posted on Wednesday, October 22nd, 2008 at 10:15 am and is filed under Javascript, Uncategorized, YUI, Yahoo!. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

3 Responses to “Creating an Autotagger with Yahoo’s Term Extraction Service and YUI”

  1. Andres Says:
    October 22nd, 2008 at 10:28 pm

    Very cool! Can really use this right now for a CMS tool I’m building.

  2. David Says:
    October 28th, 2008 at 7:39 am

    Nice! Yeah, tagging is a bit of a pain. Love the example you posted - that sells it. Now if there was only a way to embed the example in your post :). I wonder if there’s a way to hack Wordpress to display iframes?

  3. In the Wild for October 30, 2008 » Yahoo! User Interface Blog Says:
    October 30th, 2008 at 5:19 pm

    […] the use of one of YUI’s most popular developer tools, Julien Lecomte’s YUI Compressor.Creating an Autotagger with Yahoo’s Term Extraction Service and YUI: James Long wired the Yahoo Term Extractor API to a YUI Rich Text Editor to create an autotagging […]

Leave a Reply