I've been using the Bing Search API for a while on my peter.hahndorf.eu site. They site is totally static, there are no server-side components. So a search had to be implemented in JavaScript. The code was relatively simple and Bing gives you an unlimited number of search queries which I liked.

However recently I noted that the API calls didn't return any results at all, while the bing.com site itself returned the results as expected. I looked around a bit online and found out, that the API is not using the exact same data source as the web site and that several people had problems with their results.

So I turned to the other search guys. I had used an older Google API on the server a while ago but this time I wanted to replace the client side implementation.

I did not just want to place a Google create HTML fragment search box on my site, I wanted to get just the search results via JSON without any Google goo around it.

You can see the working search at peter.hahndorf.eu/search.html

They have something called the 'JSON/Atom Custom Search API', and I will describe how to use it:

To start with, you need a Google account, quiet likely you already have one. Next you need to define your own custom search engine. This allows you to search just your site not the whole web. Go to http://www.google.com/cse/manage/all and create a new search engine, follow their steps and you receive a 'Search engine unique ID' which you later need in your code.

Next you need an API key. Follow the instructions at http://code.google.com/apis/customsearch/v1/getting_started.html#get_account, the API key is the second parameter that you have to provide to Google every time you use the API. The free API plan only allows you 100 queries per day, which is enough for my small site but for bigger sites you have to pay them.

You can now test your custom search by just putting the following in your browser's address bar:

https://www.googleapis.com/customsearch/v1?key=INSERT-YOUR-KEY&cx=INSERT-YOUR-ENGINE-ID&q=YOUR-SEARCH-TERM&alt=json

You should get a page with search results formatted in JSON.

Building a search page:

Let's start with the html:

<input type="text" id="txtSearchTerm" size="40" />
<button id="btnSearch" style="display:none;">Start Search</button>

<noscript>JavaScript is required for this page.</noscript>

<div id="searchResult"></div>

<div id="output"></div>

<div>
    <a href="#" id="lnkPrev" title="Display previous result page" style="display:none;">Previous</a> <span id="lblPageNumber" style="display:none;"></span> <a href="#" id="lnkNext" title="Display next result page" style="display:none;">Next</a>
</div>

Let's see, we start with an input field and a button. Because without JavaScript our search doesn't work we hide the button and display a message to the user using the 'noscript' tag.

Next we have three divs, the first 'searchResult' is to display a search result summary or an error message. The second one 'output' is for the actual results and the third one is for the navigation to the next or previous pages. The API returns up to 10 records per call and up to 110 records in total. I did not implement a number of links to jump to the result pages directly. I feel to 'Next' button is good enough.

We also need to include our script file and jQuery, I always put these at the end of the page:

<script type="text/javascript" src="jquery.js"></script>


<script type="text/javascript" src="googlesearch.js"></script>

When developing the script you will do a lot testing and may run into the 100 queries per day limit, to work around this I created a local dummy result page. Use the url from before but instead of the &alt=json parameter at the end, use &callback=SearchCompleted. Also make sure that the search term you are using returns more than 10 results.

https://www.googleapis.com/customsearch/v1?key=INSERT-YOUR-KEY&cx=INSERT-YOUR-ENGINE-ID&q=YOUR-SEARCH-TERM& callback=SearchCompleted

now you get a slightly different result which is JSONP, it wraps the JSON data in a JavaScript function call. Save the page into a text file named dummy.js on your development server. Rather than calling Google for the results, we will now use this file.

Create a 'googlesearch.js' file for our logic, start with the document ready function:

$(function ()
{
    $('#btnSearch').show().click(function () { Search($("#txtSearchTerm").val(),0);});
    $('#lnkPrev').click(function () { Search($("#txtSearchTerm").val(),-1); });
    $('#lnkNext').click(function () { Search($("#txtSearchTerm").val(),1);  });
});

We just set up the event handlers for the links and the button which we also unhide, the second parameter 0,-1 and 1 is for paging.

Next create a search function which initiates the API call:

function Search(term, direction)
{
    url = "http://localhost/dummy.js?callback=?";
    $.getJSON(url, '', SearchCompleted);
}

The url points to our dummy file, notice the callback=? part as a parameter. It surely doesn't make any difference in our static file what parameters we call it with but we still need it. It tells the jQuery Ajax magic to treat the result as JSONP and execute it after receiving it. Naming the dummy file *.js tells the web server to send the file content with a Content-Type header of 'application/x-javascript', which is required for JSONP.

So what actually happens here? Just doing an AJAX call for the JSON data from Google does not work because we can only do AJAX calls to our own domain, not a different one like Google.com. To work around this JSONP wraps the data in JavaScript because the <script> tag in html allows the src attribute to point to a different domain. JQuery executes the JavaScript received from the server. It does nothing else than calling our callback function passing in the JSON data as the only parameter. You can see that in our dummy.js file.

The simplest version of the SearchCompleted function would look like this:

function SearchCompleted(response)
{
    var html = "";
    for (var i = 0; i < response.items.length; i++)
    {
        html += response.items[i]. htmlTitle + "<br />";
    }
    $("#output").html(html);
}

We loop through the JSON data and build up an HTML string which we then display in our output div.

Most likely we want to do a bit more with the results, like linking back to the actual page. Look at our dummy.js file to see all the data we get from Google: ] .htmlTitle, .link and .htmlSnippet are the most useful ones.

You can look at my actual implementation to see an example of how to massage the search results before displaying them. Peter.hahndorf.eu/search.html and peter.hahndorf.eu/css/googlesearch.js

Using the real thing

Now that our results look okay, we can switch over to the Google results, we need to change the url in the Search function:

var url = "https://www.googleapis.com/customsearch/v1?key="+ mGoogleApiKey + "&num=10&cx=" + mGoogleCustomSearchKey + "&start=" + startIndex + "&q=" + escape(term) + "&callback=?";

mGoogleApiKey and mGoogleCustomSearchKey are two variables that I set elsewhere with my real values. The start parameter is needed for paging and we again need the callback=? to tell JQuery to do its JSONP magic.

Paging:

So far we always only get the first 10 results. So lets add some basic paging. Normally this include quite a bit of logic, luckily Google provides some help. If you look at the dummy.js file, you see 'nextPage' and 'request' under 'queries'. Here we can see the total results in (response.queries.request[0].totalResults) and the start index for the next page (response.queries.nextPage[0].startIndex). All we have to do is checking whether there is such a value and then unhide the appropriate link and remember the StartIndex value for the next Ajax call.

Again look at peter.hahndorf.eu/css/googlesearch.js for the code.

 

My use case for this requirement was that users can add text on a web site and could include links (a tags). I want all these links
to open a new window when clicked on. So I needed to add a target="_blank" to all the a tags in the text.

My first idea was to use pure Regular Expressions, but I couldn't figure out how to use the negative lookahead feature, I needed
this because I can't simple add a target="_blank" to each a tag, what if it already has one?

So instead I am using a simple Regex to find the appropriate tags and then use a MatchEvaluator, which is a very cool
feature of the .Net regular expression class. It allows you to compute the replacement string in a method for each
match.

public class HtmlHelper
{
    string _attrib = string.Empty;

    public string AddAttribute(string source, string tagName, string attrib)
    {
        _attrib = attrib;

        string term = "<" + tagName + " [^>]+>";

        Regex r = new Regex(term, RegexOptions.IgnoreCase);

        MatchEvaluator myEvaluator = new MatchEvaluator(this.ProcessMatch);

        return r.Replace(source, myEvaluator);

    }

    private string ProcessMatch(Match m)
    {
        string tag = m.Value;
        if (tag.IndexOf(_attrib) == -1)
        {
            tag = tag.Replace(">", " " + _attrib + ">");
        }

        return tag;
    }
}

This can then be used like this:

HtmlHelper helper = new HtmlHelper();
htmlText = helper.AddAttribute(htmlText, "a", "target=\"_blank\"");

 

As I use more and more JavaScript and Ajax on my sites the number and the size of the JavaScript include files grow. What do to about this? jQuery comes in a uncompressed and minimized version, ideally I want to use the first on my Dev-box and the second on the production server. Asp.net 3.5 SP1 comes with the feature CompositeScript which allows us to combine several script into one download. Well, I am trying to get away from the asp.net ScriptManager, so I came up with my own solution.

The first version supports the following:

During development the fullsize script files are used

In Production mode, a single minimized file is used that includes all the JavaScript code needed for the page.

This file is not generated on the fly, instead it is stored on the file system. This makes it more scalable.

Here’s how in works:

All my common JavaScript include files are in one location /css/. Instead of referencing them with

<script src="../css/jquery.js" type="text/javascript"></script>
<script src="../css/topas.js" type="text/javascript"></script>

On the page, I just have a single literal control:

<asp:Literal runat="server" ID="litSciptIncludes"></asp:Literal>

Which results in the following html:

<script src="../css/lib/3.js" type="text/javascript"></script>

3.js has the compressed content of both jquery.js and topas.js

In the Page_Load method in code behind I have some code to add scripts:

ScriptIncluder inc = new ScriptIncluder();
inc.AddScript(ScriptIncluder.Scripts.jQuery);
inc.AddScript(ScriptIncluder.Scripts.Topas);
litSciptIncludes.Text = inc.GetIncludes();

The ScriptIncluder class is responsible for building the <script src…> tags. We add all the scripts we need on the page with the AddScript method, a fixed number of scripts are defined in a enumeration.

[Flags]
public enum Scripts
{
  jQuery = 1,
  Topas = 2,
  TopasDate = 16,
  TopasEdit = 32,
  jQueryUI = 256,
  jQueryMaskedEdit = 512,
}

public void AddScript(Scripts scriptID)
{
  _scripts.Add(scriptID);
  _scriptCode += (int)scriptID;
}

In my application framework I have a setting in the web.config which defines the Debug-Mode, when in development I just loop through all the added scripts and return one <script> tag for each added script. However when in production, I return a single script tag that points to a single file.

public string GetIncludes()
{
    string html = string.Empty;
    string start = "<script type=\"text/javascript\" src=\"" + Core.Config.VirtualRoot + "css/";
    string end = "\"></script>" + Environment.NewLine;

    if (Core.Config.CurrentDebugLevel >= Core.Config.DebugLevels.SharedDev)
    {
        // include plain text files
        foreach (Scripts script in _scripts)
        {
            html += start + _scriptNames[script] + end;
        }
    }
    else
    {
        // include compressed file
        html = start + "lib/" + _scriptCode + ".js" + end;
    }
    return html;
}

_scriptNames[] is a generic list of all the script file names, _scriptCode is the sum of all the scriptIDs as defined in the Scripts enumeration.

So where do all the /css/lib/nn.js files come from that are needed for the single compressed file?

I use a batch file to create them:

pushd \websites\foo\wwwroot\css
jsmin.exe lib/769.js jquerymin.js jqueryuimin.js jquery.maskedinput.js
jsmin.exe lib/819.js jquerymin.js jqueryuimin.js jquery.maskedinput.js topas.js topas.date.js topas.edit.js
jsmin.exe lib/49.js topas.js topas.date.js topas.edit.js
jsmin.exe lib/3.js topas.js jquerymin.js
jsmin.exe lib/1.js jquery.js
popd

In theory I need hundreds of these /lib/nnn.js files, but in practice there are only a few combinations that I use. What is jsmin.exe? It is a small tool based on Douglas Crawford’s jsmin (http://www.crockford.com/javascript/jsmin.html). He has C# code to minimize a single JavaScript file. I added some code to process any number of files into a single file. I can just run the batch file every time I have changed any of my JavaScript include files and can also make it part of my build process.

Any drawbacks? If every page uses a different combination of include files, all these combinations have to be downloaded once and then cached, so the actual jQuery code may be downloaded several times.

 

I tried to add the Microsoft Live Search API 2.0 to my site and ran into a couple of problems.
I used the JSON sample at \Live Search API 2.0 SDK\Samples\JSON\WebSample.html
the sample worked fine as is, but when I added a site:mysite.com like this on line 21

"&Query=peter site:mysite.com"

it stopped returning any results, even though the same query on search.live.com works.
After playing around for a while, I figured out I had to remove

 + "&Web.FileType=DOC"

and while I was at it, I also removed:

 + "&Market=en-us"

now the searches worked.

The second problem was when they were really no search results, I got a JavaScript error:

results is undefined
(response.SearchResponse.Web.Offset + results.length)

on line: 84
To fix this, I added some code around the results display:

   if (response.SearchResponse.Web.Results == null)
   {
        resultsHeader.innerHTML = "No results found";
   }
   else
   {
        // display results
        ...
   }