We ran into a situation where our CRM 4 database would take days to upgrade to 2011. It turns out we had accumulated over 150GB of attachments from CRM tracked emails. In order to drastically speed up the migration process, we decided to truncate ActivityMimeAttachment the table prior to the upgrade process. I then wrote SQL scripts to pull the data over after the conversion using the linked server functionality in SQL Server. The advantage is that data can be pulled even while one or both of the systems are in use.

If you’re not familiar with linked servers, it’s a useful feature that allows you to set up one SQL Server database to connect to another and access both within the same query or script. To link a server, you’ll want to go to the destination server, and log in. You can actually link either direction, I just chose to write the script for the destination server. The source server you’ll be linking must have the proper permissions set up. In this case, since the migration process was temporary, so I created a temporary sysadmin account. I was then able to link the source server with this SQL (if it’s not obvious, there are some tokens you need to substitute):

Exec sp_addlinkedserver '{source_server}', 'SQL Server';
Exec sp_addlinkedsrvlogin '{source_server}', 'false', null, '{source_server_user}', '{source_server_password}';

Once the servers are linked, you can start pulling over the data. This is where things get really hairy. In CRM 4, the ActivityMimeAttachment table stored all of the attachments and the associated metadata. In CRM 2011, this information is split apart, and there are extra columns that we need to supply data for. First, we’ll need to populate the Attachments table:

Insert Into Attachment
Select Body, [Subject], FileSize, MimeType, [FileName], Null as 'VersionNumber',
	ActivityMimeAttachmentId as 'AttachmentId'
From {source_server}.{source_crm_database_name}.dbo.ActivityMimeAttachment

After that, we can pull over the metadata. One piece of information that we’re missing is the “solutionId”. I sent a test email from 2011 and inspected the row it inserted to get the solutionId.

Insert into ActivityMimeAttachment
Select AttachmentNumber, ActivityMimeAttachmentId, Null as [VersionNumber],
	{solution_id} as SolutionId, ama.ActivityMimeAttachmentId as 'AttachmentId',
	Null as 'SupportingSolutionId', ActivityTypecode as 'ObjectTypeCode',
	0 as 'IsManaged', 0 as 'ComponentState', '1900-01-01 00:00:00.000' as 'OverwriteTime',
	(newid()) as 'ActivityMimeAttachmentIdUnique', ama.ActivityId as 'ObjectId'
From {source_server}.{source_crm_database_name}.dbo.ActivityPointerBase apb
Join {source_server}.{source_crm_database_name}.dbo.ActivityMimeAttachment ama
	on ama.ActivityId = apb.ActivityId

When you’re ready to unlink the source server, use the drop commands:

Exec sp_droplinkedsrvlogin '{source_server}', null
Exec sp_dropserver '{source_server}'

At this point, I was concerned that the script may crash and/or run out of memory. To avoid this possibility, I decided to pull the data over in batches. I won’t get into the specifics, but you can reference my complete script below. From a high-level, I’m adding a column to the source table to mark rows that I’m pulling over, and what the migration state is. I made it nullable so that it doesn’t affect normal CRM operations.

Declare @LinkedServerUser Varchar(Max)
Declare @LinkedServerPassword Varchar(Max)

Set @LinkedServerUser = {linked_server_user};
Set @LinkedServerPassword = '{linked_server_password}';

Print 'Linking Remote Server...'
Exec sp_addlinkedserver '{linked_server}', 'SQL Server';
Exec sp_addlinkedsrvlogin '{linked_server}', 'false', null, @LinkedServerUser, @LinkedServerPassword;
Go

/*
Create a column in the source database to track which rows have been converted.

Values:
--1=Queued to move to attachments
--2=Moved to attachments
--3=Queued to move to mime attachments
--4=Fully migrated
*/
EXECUTE {linked_server}.{crm_database_name}.dbo.sp_executesql
	N'Alter Table ActivityMimeAttachment Add T_Processed Int'
Go

Declare @SolutionId UniqueIdentifier
Set @SolutionId = '{solution_id}';

Declare @Rows Int

Print 'Copying attachments...'
Set @Rows = 1
While(@Rows > 0)
Begin
	--Queue up records we haven't moved
	Update Top (1000)
	{linked_server}.{crm_database_name}.dbo.ActivityMimeAttachment
	Set T_Processed = 1
	Where T_Processed Is Null

	Insert Into Attachment
	Select Body, [Subject], FileSize, MimeType, [FileName], Null as 'VersionNumber',
		ActivityMimeAttachmentId as 'AttachmentId'
	From {linked_server}.{crm_database_name}.dbo.ActivityMimeAttachment
	Where T_Processed = 1

	Set @Rows = @@RowCount
	RAISERROR ('Attachment Rows Updated...', 0, 1) WITH NOWAIT

	--Mark the records as moved
	Update {linked_server}.{crm_database_name}.dbo.ActivityMimeAttachment
	Set T_Processed = 2
	Where T_Processed = 1
End

Print 'Copying attachment metadata...'
Set @Rows = 1
While(@Rows > 0)
Begin
	--Queue up records we haven't moved
	Update Top (1000)
	{linked_server}.{crm_database_name}.dbo.ActivityMimeAttachment
	Set T_Processed = 3
	Where T_Processed = 2

	Insert into ActivityMimeAttachment
	Select AttachmentNumber, ActivityMimeAttachmentId, Null as [VersionNumber],
		@SolutionId as SolutionId, ama.ActivityMimeAttachmentId as 'AttachmentId',
		null as 'SupportingSolutionId', ActivityTypecode as 'ObjectTypeCode',
		0 as 'IsManaged', 0 as 'ComponentState', '1900-01-01 00:00:00.000' as 'OverwriteTime',
		(newid()) as 'ActivityMimeAttachmentIdUnique', ama.ActivityId as 'ObjectId'
	From {linked_server}.{crm_database_name}.dbo.ActivityPointerBase apb
	Join {linked_server}.{crm_database_name}.dbo.ActivityMimeAttachment ama
		on ama.ActivityId = apb.ActivityId
	Where ama.T_Processed = 3

	Set @Rows = @@RowCount
	RAISERROR ('ActivityMimeAttachment Rows Updated...', 0, 1) WITH NOWAIT

	--Mark the records as moved
	Update {linked_server}.{crm_database_name}.dbo.ActivityMimeAttachment
	Set T_Processed = 4
	Where T_Processed = 3
End

Print 'Dropping temp column'
EXECUTE {linked_server}.{crm_database_name}.dbo.sp_executesql  N'Alter Table ActivityMimeAttachment Drop Column T_Processed'

Print 'Unlinking remote server...'
Exec sp_droplinkedsrvlogin '{linked_server}', 'sa'
Exec sp_droplinkedsrvlogin '{linked_server}', null
Exec sp_dropserver '{linked_server}'

Print 'Migration complete.'

A few years ago, I set up a privately hosted Subversion repository to hold the source code for all of my various non-work related projects. These days, it is showing it’s age, as the world and my professional life has moved on to Distributed Source Control (DVCS). I decided it was time to convert to something more modern, maintainable, and upgradable in the future.

In this post, I’ll show you how I converted my repositories over to Mercurial to archive them in Kiln, complete with history. There are a number of ways to accomplish this goal, and I thought this method was fairly simple and straightforward.

Instructions

First, install TortoiseHg if you haven’t already done so. You actually won’t need Subversion installed to do the conversion.

Next, right click anywhere in an Explorer window, select TortoiseHg->Global Settings:

Go to the "Extensions” section, and check the “convert” checkbox, which will enable the conversion extension functionality:

One Repository vs Many

In Subversion, there were two common repository structure camps. One that used a single repository for multiple projects, and those that used a separate repository for each project. Due to the complexity of setting up projects on the Subversion server, I always stuck with the “one giant repo” model.

In Git, repositories will be distributed and are ideally as lightweight as possible. The typical pattern is to create a new repository for each project. This obviously means that we have to do some work to separate out a single Subversion repository into multiple Git repos.

Creating a Filemap (optional)

If you have separate repositories in Subversion, you can skip this step.

Create a “filemap”, which is just a text file with the name of your choosing that will look like this:

include "Utilities/ProjRefToDll"
rename "Utilities/ProjRefToDll" .

The first line is telling the converter to only include the path specified. The second line is telling the converter to move the project to be in the root (note the period) of the new repository. If we leave this out, we’ll only have the project we’re looking for, but our source code will be nested under folders from our old structure.

You’ll need to update this file for each project you convert.

Convert!

Run the following command line with the correct paths (omit the filemap parameter if converting the whole repo):

"<path-to-hg>\hg.exe" convert -s svn "<path-to-svn-repo>" --filemap "<path-to-optional-filemap>"

Here is the command I ran:

It will then iterate through every revision in the source repository, even if it’s not relevant to the filter you’re applying. It will create a new folder next to the folder you’re converting, but with a “-hg” suffix, like this:

Open it up, and it will be a full-fledged Mercurial repository. Do an “update” to confirm that everything looks like you expect:

At this point, you have successfully converted a project from your Subversion repository into a Mercurial repository, revisions and all.

Personally, I’m using Kiln as my online source archive. Even if you aren’t interested in Kiln, and instead want to use something like GitHub, no problem, push it up to Kiln, and then pull it via Git for an automatic conversion at any time! I’m using Kiln for the time being, considering that it’s free for up to 2 users. It works great for my personal projects that I don’t want public on GitHub. My plan is to eventually scrub those projects, and put the latest revision of each out on GitHub to make available to the community.

I’m not sure if I’m switching the full repositories to Git yet, so I like the flexibility that Kiln offers. If you are interested in converting to Git directly, there is probably an easier way.

I recommend checking out the Pomodoro Technique. It’s a super simple productivity technique that is designed to improve your focus and your own productivity by eliminating distractions, and maintain concentration by taking breaks at optimal times.

The best part is that there is no book to buy, and you can get started in minutes. The Pomodoro website can get you up and running quickly. Basically, you’re focusing on a single task for 25 minutes, and then you take a 5 minute break before starting the next Pomodoro. After 4 Pomodoros, take a longer break.

Lifehacker users voted it the #1 productivity method, and I find that it’s compatible with the Getting Things Done methodology (a book I personally recommend). Will it cure world hunger? No. In fact, you can’t even eat the timer even though it looks like a tomato.

It works a lot like scrum. You create your list of tasks, and then estimate them based on the number of Pomodoros they will take to complete. Each day is like a sprint, and each Pomodoro is like a story. As you get better at the technique, you perform daily retrospectives to improve the process and the estimation. If you don’t like their pencil and paper approach, you can use Trello.

I’m definitely not an expert, but I’ve already realized some of the benefits. It’s amazing how many little distractions occur in a 25 minute period, killing productivity. Unlike a computer, we humans are absolutely terrible at switching tasks.

With an entire team, the challenge would be coordinating everyone to minimize potential interruptions. I did find some teams using it with limited success. For example, here is a detailed analysis of Pomodoro combined with Scrum:

http://www.devoteddeveloper.com/2012/02/pomodoro-scrum-development-objective-i.html

Some interesting quotes from that article:

“I think expanding the Pomodoro Technique® to a whole team can prove very difficult. Probably it’s not even desirable.”

“How you work most efficient is very individual, but to be productive I think you have to feel comfortable about, and like, the way you work.”

Give it a try. Couldn’t hurt.

I recently had an opportunity to sell my original i7 920 PC, and build a 3rd generation Ivy Bridge.  My goal was to minimize power usage so that I could run it as a 24/7 media server, and maximize its encoding horsepower. After hooking this build up to a Kill-A-Watt, it measured at just over 30 watts at idle, and less than 100 watts maxed out.

Total Price: $897.36 + shipping

Processor: Intel Core i7-3770K 3.5GHz

This was the first component I picked out, as a suspected all of the other components would be dependent on the processor. We have built some AMD machines at work, but they are severely lagging behind their Intel-based counterparts for performance and power usage. Intel is a little pricier, but definitely worth it. I went on the high-end as far as speed to minimize video encoding time, and hopefully prolong my next upgrade as long as possible.

Since I don’t have any intentions of using this system for high-end gaming, my goal is to actually use the Intel 4000 graphics that are integrated into the chip. Integrated graphics are still significantly behind their discrete counterparts, but this saves me over 100 watts of power usage.

Motherboard: ASRock Z77 Extreme 4

I always find the motherboard the hardest component to spec for a new build. In the past I’ve chosen Gigabyte boards for their low price and innovative features such as long-life solid state capacitors. However, I ran into some buggy firmware with my last build, and hoped to avoid them this round. Solid state capacitors are now the norm, so features and stability were my primary goals. We’ve used a few ASRock boards for builds at work, and they have worked flawlessly. They are also the current leader for fast booting when using UEFI. At $105 for this board, it seemed like a great price/value ratio. It has dual-monitor outputs so that I can use the integrated graphics while still keeping 2 monitors.

Below is a thermal image from the board. Notice that the ASRock chipset is at 143F. I can’t explain why it runs so hot. It’s not hot enough to be a huge concern, but I would prefer to keep every component cool for long life.

Memory: 32GB G.SKILL

Memory speed used to have a significant impact on the real-world performance of your system. These days, buy memory that is well reviewed and falls within the memory speed requirements of your motherboard. At the time of my purchase, I was able to get the Ares memory for only $55 for 2x8GB strips. At this price, it was worth maxing out the board at 32GB (I’m still in awe) just to avoid a future upgrade. One interesting feature of the motherboard is the ability to use part of the RAM as a RAMDISK, which is essentially a virtual hard drive that would outperform even the best SSD.

PSU: SeaSonic M12II 620 Bronze 620W

Bad power supplies create bad power, which kills components. A good power supply will provide clean power, which will make your system more stable, and will help ensure a long lifespan for all of the other components.

Why did I choose a 620W power supply? My experience has been that people often oversize their power supplies. If you use a 1000W power supply on a 50W load, it’s going to be terribly inefficient. Power supply efficiencies, which measure the amount of power lost in the conversion from AC to DC, are only accurate when the power supply is under heavy load. As you get out of the sweet spot, efficiency drops. When sizing a power supply, the #1 power consumer is your video card. An inefficient video card will pull more power than the rest of your system. Add multiple cards and the power requirements go through the roof. In my case, I’m using integrated graphics and shooting for low power usage. My goal was to stay under 100W at load on the DC side of the power supply. With an 85% efficiency, that would be about 118 watts on the AC side. Even at 620W, this power supply is WAY oversized. However, I have a quality power supply that gives me plenty of headroom for any upgrades down the road. Besides, there are not many well-built power supplies at lower wattages these days.

Hard Drive: Crucial m4 128GB SSD

SSD’s are undoubtedly the biggest technological breakthrough in computing as it relates to the perceived speed of the computer. People usually quote 7 second boot times, but the value is truly in the fact that applications open instantly.

I’ve had my fair share of bad luck with SSD’s. Since they’re a newer technology than their spinning disk counterparts, they have firmware that can be buggy in certain versions. I was a victim of the OCZ firmware bug, as well as the 7 month bug in the Crucial drives. That being said, I trust this drive thanks to its fairly consistent track record, and favorable performance.

Why only 128GB? I have a 3TB storage array that I use for my bulk data. Unless you install a number of large games on your SSD, or store other bulk data, 128GB is fine for just about everyone. If you feel more comfortable with the 256GB version, the price gap has closed significantly.

Case: Antec 300

I hate choosing a case. Without seeing them in person, it’s difficult to gauge their build quality and features. I’ve used this case in the past, and it’s worked well enough. It’s usually inexpensive, and easy to work with. It has excellent ventilation, a cleanable air filter, and a place to mount your SSD drive.

CPU Fan: ZALMAN 92mm

I ran some temperature tests on the i7 Ivy Bridge with a stock cooler, and found that it can run pretty hot under load. An upgraded fan was probably not absolutely necessary, but it’s a $50 investment that guarantees that your CPU will run 20-30c cooler. This one is a simple design that’s easy to install, and the copper fins dissipate heat very easily.

Here is a photo using a Flir thermal camera. Notice that there is almost no measurable heat coming from the heat sink on the right side of the image.

Blu-ray / DVD: LG Blu-ray Drive

This was probably overkill, but it was the only Blu-Ray drive that I could find that got exceptional reviews, and is 16x.

I recently spoke to some of the wonderful guys on the ASP.NET team over at Microsoft (@shanselman @haacked and others). Somebody asked if anyone had any complaints they wanted to share. I brought up the topic of Silverlight since it’s been on my mind recently.

I would like to make it clear that the product we built with it is still amazing, and Silverlight has a lot to do with that. Silverlight is a solid product that has reflected the amount of effort that the developers put into it.

When I was originally confronted with the decision of building our user interface in Silverlight or HTML, I chose Silverlight for the following reasons:

• It’s the cool thing to do
• Perceived fast development
• User interface performance without having to touch “icky” JavaScript

Now that the first commercially available version of our software is available, I can’t help but admit some regret with the choice to use Silverlight.

The bottom line for me comes down to the fact that HTML runs everywhere, and Silverlight runs some places.

It’s becoming abundantly clear that the future of computing is in a wide variety of form factors, and they’re getting smaller. iPads, tablets/slates, iPhones, Android Phones, Microsoft phones, etc. are all widely available. Even Microsoft’s own Windows CE devices don’t support the Silverlight we know and love. These new types of mobile devices are showing us that the future it not just staring at a 19” screen. Our own Silverlight front-end is not only un-optimized for these devices, but worse yet, it doesn’t work, period.

So is the HTML story better? I believe it is. Let’s examine my original reasons for going to Silverlight:

• It’s the cool thing to do – I believe this gap is closing. jQuery and other animation and scripting tools are making cross platform development reliable and easy to develop. The proliferation of web standards has been helpful as well.
• Perceived fast development – My mistake here was thinking that since we didn’t have to deal with HTML, CSS, and JavaScript issues, we would be able to develop much quicker. The reality is that the time we would have spent on those issues is now spent dealing with another layer of misdirection. We now have to get the data from our database to the web server, but then we have to get it to the client. This is all while managing the subtleties of RIA services and LINQ to SQL objects. ASP.NET MVC removes one layer of mapping and complexity (we can optionally add layers obviously).

  On notable exception to this is our scheduling interface. We’re using the wonderful Telerik scheduler, which really shows when Silverlight can shine compared to an equivalent HTML page.

• User interface performance – Silverlight wins this one. We’re doing some amazing things with layout transforms that really wows our customers. The easy animations will be about the same amount of work in JavaScript, but the more complex animations will likely be easier in Silverlight.

Most of my reasons make common scenarios a wash (or possibly in favor of Silverlight). However, there are reasons that I’m avoiding Silverlight going forward:

• It doesn’t run well on Linux, CE, iPad, etc – This is huge. Unless you have a captive audience, you’ll want to make sure that your application runs on the newest devices. HTML is the only sure bet.
• Memory leaks/issues – This one really irks me. I’m sure that there are issues in our code that cause some leaks, and I’m sure some can be attributed to this seemly unsolvable issue. We have a screen that stays open on a touchscreen, and it consumes all the memory on the machine within half a day. The bottom line is that these simply don’t happen in HTML.
• Initial load hit – I know there are ways to mitigate this in Silverlight by spreading your application over multiple XAP files. However, HTML is the better choice for slow connections. The slow speed is spread throughout the screens, instead of switching between fast and slow.

In conclusion, all of the amazing reasons that I went with Silverlight have faded, and now I’m left with the reality of a platform that will never achieve 100% accessibility. There are plenty of situations where Silverlight may be the perfect fit, but next time I wont be as quick to lean toward Silverlight without serious consideration as to the potential audience and goals.