• www.test.com/path
• www.test.com/path/

Search engines could potentially index both paths, and count them separately in their search results. The first solution is to make one of the versions return a 404 (page not found) response code. Hopefully, this would deter anyone from linking to the non-preferred version. We can do this by creating a custom route constraint, by implementing the IRouteConstraint interface:

public class TrailingSlashConstraint : IRouteConstraint
{
	private readonly bool _trailingSlash;

	public TrailingSlashConstraint(bool trailingSlash)
	{
		_trailingSlash = trailingSlash;
	}

	public bool Match(HttpContextBase httpContext, Route route, string parameterName,
		RouteValueDictionary values, RouteDirection routeDirection)
	{
		return _trailingSlash == httpContext.Request.Url.LocalPath.EndsWith("/");
	}
}

Then, when you define the route, supply an instance of the constraint:

routes.MapRoute("mobileVersion", "m",
	new { controller = "home", action = "mobile", id = "" },
	new TrailingSlashConstraint(false));

The second solution is to redirect the non-preferred version to the preferred version. We can do this by either creating a custom HTTP Module, or by using a canned product like the free UrlRewriter.NET.

Should I use a trailing slash or not?

This is one of the problems with MVC. In ASP.NET webforms, you could use a default page to avoid having to specify a file in the URL. This gets you the nice MVC style paths. The good part was that if someone accessed the version without a trailing slash, it would automatically redirect to the version with the trailing slash. So in ASP.NET WebForms, the trailing slash was obviously preferred since it was the default behavior.

Now, in ASP.NET MVC, they decided to make the default behavior the opposite. It does serve up both versions as I mentioned, but when it generates links, they don’t have a trailing slash.

If you’re long term solution is to use MVC, I recommend dropping the trailing slashes, just to avoid the pain. I’m really hoping that by the time the final version of MVC comes out, they’ll give us an option to define routes that should have a trailing slash.

Duplicate Content

Duplicate content is a major issue. If a search engine (Google, which we’re primarily concerned with) finds multiple identical pages, it could be seen as a spam technique. Google likes original content, and penalizes duplicate content.

The problem is that the ASP.NET MVC default routing is too forgiving. If I have a page with this address: “/controller/action/id”, the routing engine happily serves it up at “/controller/action/id/”. There is no reason to not be strict on this. In ASP.NET WebForms, if you forget the trailing slash, it will automatically perform a 301 (permanent) redirect to the version with the trailing slash.

ASP.NET MVC has a bug (I’m calling it that) that won’t let you define a URL as requiring a trailing slash. Below, I’ve defined a route as a sample. In the URL to match, I have a trailing slash. In the routing code, the trailing slash is removed when it’s added to the routing table. This also has the side-effect of generating the URL’s without a trailing slash.

routes.MapRoute(
	"Legacy-Firefox",
	"Firefox-Extension/",
	new { controller = "Home", action = "Firefox", id = "" } );

Since routes can be configured to reuse actions and controllers, it makes more paths to follow. If I again use the route defined above, I end up with all of these valid addresses that could potentially be linked to and indexed:

• a.com/Firefox-Extension
• a.com/Firefox-Extension/
• a.com/Home/Firefox
• a.com/Home/Firefox/
• a.com/Home/Firefox/anythingyouwant

If you’re lucky, Google won’t penalize duplicate content. However, if Google indexes the same content using multiple URL’s, you won’t get the benefit of focusing the PageRank. A similar situation occurs with you have a site that can be addressed like “cnn.com” and “www.cnn.com”. They counted as separate pages, that end up competing for good rank.

Legacy URL’s

I’m sure there is a large group of people that are anxiously awaiting to take advantage of the new MVC style development. Many of them will undoubtedly have existing URL’s they want to preserve.

There are a couple of ways to handle this issue. The search engines would prefer that your URL’s simply remain the same. This is possible, but requires some fancy routing. The SEO community highly recommends this approach (with good cause).

Another way to handle it is to adopt the new REST style URL’s that typically make the most sense with an MVC approach: “/controller/action/id”. Then, setup 301 redirects from the old addresses to the new one’s. This article discusses the technical details. In theory, this should be the best scenario. However, Google themselves say to get the incoming links pointed to the new addresses ASAP. The truth is, this solution sucks. I’ve actually done this with a site. It was search engine suicide for a couple of months. I eventually got my old position back, but lost a significant amount of revenue because of it.

Yet another way that I found is to set up multiple routes so that the content is accessible with both the old and new addresses. If you’ve been paying attention, you’ll know that this counts as duplicate content, and is very, very bad. I was in shock when I found this approach being advocated.

Conclusion

I’m not saying the routing system is completely wrong, I just think it would be set up so that the easy way of migrating a site is the correct way, or as close to it as possible. I don’t want to have to write custom routing. At the very least, come up with a way to designate that a particular action has a single path (and cancel out additional paths in other routes). It would also be nice if there was a way to use the old style urlMapping section in the web.config for legacy URL’s.

If I’m completely wrong about how the routing works, let me know. It’s difficult to find good information (which is understandable right now), and I’m admittedly still in an early learning stage.

One of the big arguments against heavily using flash on your site was that it certainly wouldn’t help you in Google. Now, many will see that argument as being gone. We can now look forward to more annoying flash content, and maybe even some flash that is used correctly.

With great power comes great responsibility

-Stan Lee

If you think you can now cheat and use flash content instead of HTML, you’re probably wrong. There are many questions that have now been raised (some have been answered):

• Does each flash file count as a page?
• Do links to and from flash content count for PageRank?
• Do you really want ALL of the text in your flash files indexed?
• Does the SWF get executed in any way, so that the generated text can be indexed?

It’s going to take a while before Google gets good at indexing flash files. It’s also going to take some time for people to really understand how the process is working. I wouldn’t be too quick to convert something to flash just because Google can see it now.

Only use flash when it makes sense to your users. For example, Choice Shirts has an HTML website, but their shirt designer is flash (Flex actually). If their designer doesn’t get indexed, it’s not a big deal. The designer is there for usability, not for the search engines.

Our resident SEO expert spends nearly every day researching the current trends in SEO. If you try to do the same, and you’re the only one running your site, you won’t have any time to improve your site and come up with new content. You need to determine the right balance of time to make your site a success.

• Define your goals – The first thing you need to do is ask yourself what the purpose of your site is:
  • Are you trying to sell something online?
  • Are you trying to get leads?
  • Is your site simply a place you direct people to for more information?
• Focus on Google – Google is the by far, the most popular search engine. Most likely, you’ll want to focus your attention on them.
• Make life easy for Google – If you expect Google to make decisions, you might not like the results.
  • Avoid duplicate content – If you have the same content on more than one page, you may be penalized. At the very least, Google won’t know which content to give you credit for.
  • Make your HTML as simple as possible. A well formed XHTML design will go a long way.
• Optimize the way your pages look in the search listings – Google will only show the first 60 or so characters in your title, so keep that in mind. Make sure you DO use meta description tags in your pages, because Google WILL use those in it’s listings. Remember, you’re not just optimizing for the Google index, you also want your titles and descriptions to entice users to click on your result.
• Write original content – Google is looking for original content, and it has often been said that content is king. Even if Google isn’t giving you points for original content, Google is looking for information that is relative to the times. If you’re consistently writing good, original, and up-to-date content, Google will eat it up.
• Create a sitemap – Sitemaps allow you to give the search engines a "map" to all of your pages, which ensures that they’ll be found as quickly as possible. It also gives you an opportunity to specify how often pages change. If you’re using Wordpress, you can use an automated sitemap generator.
• Get relevant links – Google is built on a reputation system. When reputable, related sites link to you, Google will realize the popularity of your pages. The best way to get good links is to write good content. Another way is to make friends in the community, that have similar interests. They might be willing to add your site to their blogroll, or mention you in their content.
• Tell Google what to NOT index – You can create a "robots.txt" file that tells the search engines what they can, and cannot index. You want to filter out all of the pages that are not original content. For example, on this blog, I filter out everything except the front page, about page, and the posts themselves. You can see my robots file here. I have talked about using a robots.txt file to avoid duplicate content in the past.
• Use the Google webmaster tools – If Google is having problems indexing your site, you’ll certainly want to know why. I check the webmaster tools daily to make sure that there are not any problems.
• Do what Google asks – Google publishes their recommended webmaster guidelines. You definitely don’t want to do something that Google explicitly forbids. Your playing their game, you need to play by their rules.
• Have simple URL’s – Your URL’s should be human readable, and more importantly, should be simple enough that search engines won’t get confused. Avoid URL parameters, and instead opt for path or page based names. For example, http://www.yoursite.com/posts/seo-tips.html.

If you have the time to take your SEO skills to the next level, there are immense resources available. The best way to find them is to simply do a Google search. The sites that do the best job with SEO should appear at the top!

When I initially set up my blog, I set the title to "Young Technologies Tech Blog". Look what happened in the search results:

YUCK! There are two major problems here. The first is that the titles are not at all useful to a human. How would you expect anyone to click on titles like this?

The second problem is that my title tags don’t tell Google anything interesting about my site. You want the search engines to figure out the keywords in each of your pages, and having those keywords in the title reinforces that.

In "classic" blogger, you would simply use

<title><br /><mainorarchivepage><$BlogTitle$></mainorarchivepage><br /><itempage><blogger><$BlogItemTitle$></blogger> - <$BlogTitle$></itempage><br /></title>

instead of:

<title><$BlogPageTitle$></title>

This would use the title of the post item as the title of the page. The problem is the "new" blogger has a completely different template system. Be sure to back up your template before editing it! Here is what I did, step-by-step:

• Shorten your actual blog title. I was unable to figure out how to remove the blog title from each post page, without losing the post title. Unfortunately Blogger doesn’t have a token for the item post title. For my blog, I changed the title to "YTechie.com", which I think is a reasonable prefix for my post page titles.
• Customize the title tag in your template. Do this by editing the HTML, expand the widget templates, and put the following code in place of the existing title tag. This will allow you to have a custom title just for the front page.

<b:if cond='data:blog.pageType == &quot;index&quot;'>
 <title>Front Page Title (change this)</title>
<b:else/>
 <title><data:blog.pageTitle/></title>
</b:if>

• In the code above, put in the title you want for the front page. Try to keep it around or under 66 characters. For additional guidelines, consult this guide.
• Since you changed the title of your blog to something shorter, that will show up in the header as well. To customize the header text, replace this:

<b:if cond='data:blog.url == data:blog.homepageUrl'>
   <data:title/>
 <b:else/>
   <a expr:href='data:blog.homepageUrl'><data:title/></a>
 </b:if>

With this:

<b:if cond='data:blog.url == data:blog.homepageUrl'>
   This is my header title (change this)
<b:else/>
   <a expr:href='data:blog.homepageUrl'>This is my header title (change this)</a>
</b:if>

You also want Google find pages on your site that have substance, and that are not just a copy of content from one of your other pages.

So how do you avoid it on your site? The first step is to identify potential pages that have duplicate content. It’s probably happening without you even being aware of it.

Type this into Google: site:http://www.yoursite.com

I’m using blogger, and by default here are some pages that are indexed that should not be:

• http://www.ytechie.com/2008/04/aspnet-linkbutton-and-seo.html?widgetType=BlogArchive&widgetId=BlogArchive1&action=toggle&dir=close&toggle=YEARLY-1199167200000&toggleopen=MONTHLY-1207026000000
• http://www.ytechie.com/2008_03_01_archive.html

Now that we’ve identified the offending pages, we can create or modify our robots.txt file, at the root of our site.

Here is what I could add to my robots.txt to block those pages:

Disallow: /*?
Disallow: /*_archive.html$

Once you’ve updated your robots.txt file, you can use the Google webmaster tools to test it. For more information on how to edit your robots file, including syntax, consult Google.

There is one big problem. If you’re using a service like Blogger (like this blog), you can’t edit your robots file. There has been talk of adding support, but we have to deal with what is available.

The best I’ve been able to come up with, is adding this into the head (look for <head>) of my template code:

<b:if cond='data:blog.pageType == "archive">
 <meta name="robots" content="noindex, nofollow" />
</b:if>

This adds a noindex and nofollow meta tag to the generated archive pages. I have not yet figured out how to remove pages that contain parameters (?param=value). If anyone has a way to do it, please let me know! I’ve actually been considering removing the archive widget to solve it.

<a href="javascript:__doPostBack('ctl01','')">Click me!</a>

Notice that it’s simply a standard hyperlink with a JavaScript call. Typically, the search engines are only going to look at your HTML. They’re not going to evaluate the JavaScript. Doing so would be a big can of worms.

So basically, the LinkButton is going to be invisible to the search engines. At most, they might look at the words in the link text, and consider them as part of the rest of the content.

Remember, the purpose of the LinkButton to be a replacement for the ASP.NET Button control, but with the look of a hyperlink.