Archive for software

Cloud Computing (and Azure) - Right for your site?

Everyone seems to be getting excited lately about the prospect of cloud computing. Just like many others, I get excited by the idea that I wouldn’t have to worry about adding servers to scale up. Theoretically, a guy (or girl) could make the next YouTube, in his basement, for free. However, there is one huge advantage that most people ignore, and that’s the fact that’s also perfect for a small scale website.

iStock_000004135866XSmall

I’ve tried or considered many different ways of hosting my content:

  • Shared hosting - Cheaply host your sites, but be at the mercy of their IT guy messing with your computer and rebooting it for automatic updates. Also, in my experience, the performance is terrible if your traffic spikes. They typically have hundreds of users on the same server as you, and you all get to compete for performance.
  • Dedicated hosting - This is what I use now, because it ensures that I get the full performance of a machine. The disadvantage is that I have a single point of failure, and I have to manage the machine myself.
  • Hosting from home - Yes, people actually do this. If you have a high enough upload speed it shouldn’t be too bad. The problem is that your connection typically won’t be able to handle traffic spikes. You’ll also potentially be a victim of power or Internet outages, where professional hosts would have redundant systems in place (in theory).

Now, let’s talk about cloud computing. That magical cloud that many don’t understand. There are two potentially viable cloud computing methods available right now:

  • Cloud virtual machines - Amazon’s EC2 solution is probably the most popular in this category. Basically, you can create, start, and stop virtual machines remotely. You just pay an hourly rate while the computer is running. You can even upgrade and downgrade the hardware as needed. The advantage is that you can treat the computer like a physical machine and configure and use it however you like. The disadvantage is that maintaining individual machines can be time consuming and is not necessarily part of your core business.
  • Cloud application server - Instead of creating virtual machines, a cloud application server runs your application directly. You no longer worry about the constraints of a physical machine. You application could potentially be run on dozens or hundreds of servers simultaneously. The major advantage is that there is little to no maintenance, because that is the job of the provider.

I see the cloud application server as having some of the greatest advantages. You’re free to write your application with a level of abstraction, which allows you to solve the problems you really want to solve.

One of the most well known cloud application services is the Google App Engine, which currently supports Python applications. Microsoft joined the game recently with Azure for ASP.NET.

As I mentioned, not only do application servers let your applications scale up, they let you pay only for what you use. This is great for the small to medium website’s that are stuck with bad shared hosting or difficult-to-manage dedicated hosting. The fact is that most sites get a few hundred visitors daily or less. If you start to think about how often a page is actually requested, you’ll realize that it’s not very much. Even with 500 users requesting 5 pages each in a 12 hour period can easily be handled with a very low end server from years ago.

The reason that application servers are so much more efficient than shared hosting is because they’re built from the ground up to spread the load around. This results in higher utilization, but more headroom for any single application. Shared hosting providers can move users between servers, but it’s usually a manual, and often difficult process. You’re bound to a specific physical machine (unless it’s VPS hosting), and if it goes down, so does your site.

Cloud computing is also a great way to handle traffic spikes such as the Digg effect. Let’s say that you only have 500 visitors today, but might get 10, 100, or 1000 times more in a single day. It happened to FaceStat. They went from 10,000 page views per day to almost a million because of a story on the front page of Yahoo. They had to scramble to add application servers and develop a scaling strategy immediately.

Conclusion - Cloud Application Server Benefits

Cloud computing has tremendous benefits. You no longer have to worry about scaling the underlying hardware, you simply pay as you go, and you can handle traffic spikes with ease. Once cloud computing becomes mainstream and absolutely reliable, there will be few reasons to not use it.

Secure XAMPP by only allowing local access

This site and a couple others are served up on a dedicated server. To make it easy to set up Wordpress, I’m using XAMPP. In this post, I’ll give a quick overview of XAMPP, and then also show you how to secure it so that administrative utilities are only available locally.

XAMPP Super-Quick Overview

XAMPP is basically a quick way of setting up MySQL, PHP, Perl, and Apache. You can download it, extract it, and run it from any location. If you’re not experienced Apache/PHP world, this is the easiest way to get something working ASAP. In the next image, you’ll see the directory structure under your XAMPP directory:

image

Here are the key folders you would probably need to worry about:

  • apache - Contains the installation of Apache.
  • htdocs - Contains the folder that is served up by Apache. If you want to install a web application such as Wordpress, you probably want it in here.
  • mysql - Contains the installation of MySql.

Now, if you simply want to install Wordpress on XAMPP, I’m not going to write yet another tutorial. There are plenty out there, most with screenshots of step-by-step instructions, just Google for them.

Securing XAMPP

Once I configured XAMPP, I stupidly assumed that the utility applications like phpmyadmin would not be publicly available. I was very wrong, and was warned before anyone decided to do something bad.

Most of the instructions I found through Google for securing the utility paths seemed kind of weak to me. They basically work by securing those paths with a password. I’m a little paranoid, so I don’t want those paths remotely accessible at all.

The first thing I did was lock down Apache security so that it’s very restrictive by default (apache/httpd.conf):

<Directory />
    Options Indexes FollowSymLinks Includes ExecCGI
    AllowOverride All
    Order deny,allow
    Deny from all
</Directory>

Make sure that you don’t have any other directory directives in your configuration file that may override this.

Next, in the .htaccess file for each Wordpress installation, I added this line: “Allow from all”. This basically tells Apache that this folder is safe to serve up to everyone.

Now, the problem is that XAMPP has a configuration file that overrides the utility paths and allows access for anyone. To fix this, perform a search and replace in the (in Apache/Conf/Extra) “httpd-xampp.conf” file to change “Allow from all” to “Allow from 127.0.0.1″. Now, all of the XAMPP directories will only be served locally.

Conclusion

My background is certainly not Apache/PHP, but I’m still learning. If I made any mistakes in my configuration, please leave a comment or send me an email.

Vista actually is pretty good, honest

I know that Microsoft doesn’t need anyone defending them, but I think that the series of events surrounding Vista were certainly not ideal. I’ve highly anticipated every new release of Windows, and Vista was no exception.

Vista Logo

Watching the progress of Vista, I couldn’t help being disappointed. Vista was supposed to change the world. There would be an amazing new file system, a cutting edge graphics system, and .NET driving many of the core functionality. But one by one, the features got neutered or disappeared completely.

When it was released, the bad hype had already nearly killed it for me. Then I installed it in a virtual machine and learned what a pile of crap it really was. It took minutes to boot, and the UI was barely usable. Every few seconds I would get security prompts. I was done with this OS for a long time.

Fast forward to Windows 2008. I started reading great things about how this OS made a great desktop OS. At the core, it’s Vista SP1, but with much of the crap removed. I tried it out, and loved it. I had a few issues with incompatible drivers, but that’s expected when using a just-released 64-bit OS.

Then, Vista SP1 was released, and most of the cynics said it was "decent". Time had helped fix the driver issues, and SP1 addressed specific pain points. Eventually I got around to installing Vista SP1 (64 bit) over Windows 2008. I was amazed. Nearly all of the drivers were already included. It was fast, and some of the new features, while not life changers, were a boost to my productivity none-the-less.

Recently, I purchased a new laptop. It’s shocking what you can get for a mere $650. It’s a 14.1" HP (dv2910us) with 3GB RAM, 2GHz Core 2 Duo, 250GB hard drive, 802.11n, DVD burner, and even a built-in removable remote control. Of course it included a license for Vista. HP includes a shocking amount of crapware, so I reinstalled Vista from my MSDN media. One major improvement over XP is that your OEM key works with MSDN media. That means I can actually use the key that was on the sticker of the laptop! Thank you Microsoft (and it’s about time)!

It runs great on my laptop, and I’ve had no complaints whatsoever. In fact, I think I’ve had less issues than if I was running XP. Much to my surprise, a few days ago I get a prompt to install updates. The update in the list was SP1. Yes, that’s right, I wasn’t even using SP1. Maybe Vista was bad when it came out, but I think the horsepower of modern computers combined with finally decent driver support erases the bad history.

Vista Copy Dialog

I’ll sum up what I love about Vista over XP in one paragraph. Integrated search from the start menu rocks, and I use it constantly. I also love the new copy dialog, which tells me the actual speed of file transfers. As far as 64-bit support, I’ve been very impressed. It’s very possible to run the 64-bit version and have a decent change of all of your devices working as expected. This is a must since typical new high-end computers are going to start going over 4GB RAM soon (if they haven’t already). There are even hard to find features that are very nice. When you copy a folder over another folder, it actually gives you a reasonable set of options, as well as enough information to make a decision about merging, overwriting, or even renaming them.

Vista UAC Dialog 

Now what about UAC? You know, the feature that pops up an annoying dialog every 5 seconds? Well, if you run your computer as a local administrator (like you did in XP), you’ll rarely see them. I actually appreciate them because it does make your computer safer.

Conclusion

Make up your own opinion about Vista and give it a try. It’s mature enough that I recommend it to my tech friends. I was just as apprehensive as everyone else, but I’m glad I made the switch. In case you’re wondering, no Microsoft hasn’t paid me, but it sure would be nice :-).

New Backup Solution - JungleDisk + Amazon S3

I’ve settled on a new backup solution. I’m going to be using Amazon’s persistent storage solution called S3. Amazon provides a virtually infinite, scalable storage cloud that allows you to store files indefinitely. You pay a small fee to get the data there ($.10/gig), a fee to store the data ($.15/gig/month), and a fee to retrieve the data ($.17/gig).

Features I was looking for:

  • Reasonably Priced
  • Automatic
  • Reliable
  • Scalable
  • Well performing
  • Easy

Price

To automate my backups, I’m using a product called JungleDisk. You can purchase it for $20, and you get free upgrades for life. I love products that have free upgrades for life, since I don’t have to worry about when buy it. They also allow you to use it on unlimited computers, which I definitely need.

JungleDisk by default talks directly with S3, so they don’t need to run any servers. You’re counting 100% on the reliable storage provided by S3.

Organization

Amazon stores your files in "buckets", which you can think of as a single level folder/directory structure. JungleDisk can easily connect to multiple buckets at the same time. You configure each bucket independently. JungleDisk can automatically detect all of your buckets, and you can easily create new ones.

image

I highly recommend creating buckets for each logical group of files you want to back up. Try to avoid sharing a bucket between computers when possible. If you tell JungleDisk that a bucket is only used on one computer, it doesn’t have to query S3 to determine what needs to be synchronized. The default is set to multiple computers. This setting is under the "Bucket Settings" for each bucket.

image

Each bucket also lets you choose what to back up. Of course there are extensive options for backing up subfolders, excluding files, etc.

image

You can even set up how your local folders get mapped to the remote folders. This lets you do multiple folders in one bucket. On my laptop, I have a couple of folders that get backed up to a single bucket, but are organized into different folders so that I could easily restore them independently.

image 

Scheduling

Setting up a schedule is very easy. For example, on my laptop, I have it set to synchronize my files every hour. It uses the timestamp’s of the files to determine if there are any new, changed, or deleted files. Since I’m not sharing this particular bucket between machines, it can instantly determine if anything needs backed up.

image

 image

Security

JungleDisk has all the security options you would expect. You can communicate with Amazon over port 80 unencrypted, or use SSL. I actually turn off the SSL option, because I use the JungleDisk encryption. I don’t see a reason to do double encryption.

image

When you create the bucket, you can specify a custom key that encrypts your data. I like this option because I am the only one that has access to the data. Even Amazon can’t tell what I’m storing on there.

image

Performance

Since Amazon is providing the storage, they’re able to scale indefinitely. You can be confident that they can handle whatever you throw at them. They had no problem letting me upload at over 56,000 kbits/sec from my dedicated host. I backed up 4 gigs in about 10 minutes.

Fast Upload

Other Features

  • Bandwidth limiting - If you don’t want to use up all of your upstream or downstream bandwidth, you can limit it, and even schedule when it’s limited. This could be useful for limiting the connection during the day. However, I much prefer a QoS solution since it will maximize the amount of bandwidth I can use.
  • Previous versions - There are extensive options for storing previous versions of changed or deleted files. This option is very impressive, and great for documents.
  • Network drive - You can make a bucket show up as a drive on your computer, which allows you to drag and drop files to and from the bucket.
  • Jungle Disk Plus - For $1/month extra, you can get JungleDisk Plus. They use an Amazon EC2 server to proxy your data to S3. This allows you to resume large file uploads, and also lets you send just the differences. If you’re backing up large files and/or files that may have sections change frequently, this could end up saving you money.

Using an army and luck to reach critical mass

This post is going to explain the importance of your product reaching critical mass. When I say “product”, I mean an actual product you’re selling, or simply a website or blog. When I’m talking about critical mass, I’m describing the point at which your product becomes viral, sometimes known as the network effect. This should be a lesson to anyone thinking of creating their own product or service.

image

Above, you’ll see the typical technology adoption bell curve. What you need to realize is that you’re starting on the left, and you’re trying to get up the hill. Do you think it’s easy? Well, judging by Youtube, twitter, milliondollarhomepage, Digg, or myspace, it must be easy!

The truth is, you should have a path to success. Here are just a couple of paths that have worked for other products:

  • Create a product that is leaps and bounds better than anything your potential customers have ever seen - An example is Google, which was originally created by students in college. The brilliance of the algorithm and its implementation were the start of a massive company.
  • Have an army of followers that listen to your advice - A great example is Steve Jobs. Before he even makes a new product announcement, people line up at Apple stores. People trust that he’ll make cool stuff, so they listen to whatever he says. You can bet that if Steve Jobs mentioned your product, you would people lining up at your door to buy it.
  • Get lucky - It happens time and time again. Multiple products are released at the same time, all with similar features and price. Sometimes one of them gets lucky, and the others die. An example is the VHS vs Betamax format war. VHS was considered the inferior product, yet it went on to become the de facto standard.
  • Create a product that is viral by nature - Twitter and Myspace come to mind. Once one person joins, they’re begging their friends to use if. If they don’t, the service is useless. The result is that you get an army of free advertisers talk to your key demographic.
  • Spend a ridiculous amount of cash to bombard users with advertising - infomercial’s and those annoying “we’ll double the offer” commercials come to mind.

Hopefully I’m making the situation look difficult. I couldn’t find any concrete numbers, but you can be sure that more than half of online businesses fail within the first couple of years. That includes well-funded businesses. If you expect to start the next Fog Creek Software while working part time in the evenings, you need to have a plan.

The best advice I can give you is to do whatever it takes to get your product into the hands of as many people as possible. It might mean making partnerships with someone of influence, or it might mean creating a viral marketing campaign. It might also mean that you’ll have to give your product away for free, build up your army of followers, and then invent another great product. If you already have a product with a good user base, you’re probably already in good shape. If you are just starting out, don’t think that people will magically find you, unless you’re counting on the “lucky” path I described.