{"id":3234,"date":"2023-02-25T08:00:56","date_gmt":"2023-02-25T08:00:56","guid":{"rendered":"https:\/\/jamiemaguire.net\/?p=3234"},"modified":"2023-02-25T11:05:14","modified_gmt":"2023-02-25T11:05:14","slug":"handling-database-migrations-in-mature-applications-with-fluent-migrator","status":"publish","type":"post","link":"https:\/\/jamiemaguire.net\/index.php\/2023\/02\/25\/handling-database-migrations-in-mature-applications-with-fluent-migrator\/","title":{"rendered":"Handling Database Migrations in Mature Applications with Fluent Migrator"},"content":{"rendered":"<p>A while ago I had to implement a solution to automatically update multiple databases in a mature production environment as part of a CI\/CD pipeline.\u00a0 The environment and setup had some nuances.<\/p>\n<p>Getting licenses for Redgate wasn\u2019t an option, neither was rewriting code to use Entity Framework migrations.\u00a0 Batches of dynamic SQL scripts were supplied by other processes also had to be run.<\/p>\n<p>I needed a way to bundle this into a cohesive and repeatable process.\u00a0 I needed mechanisms for rolling back the process in the event of a script failing.\u00a0 I\u2019ve written this short guide as an end-to-end for future reference.<\/p>\n<p>Full code for this walkthrough can be found on GitHub <a href=\"https:\/\/github.com\/jamiemaguiredotnet\/SandboxFluentMigrator\">here<\/a>.<\/p>\n<h2>Fluent Migrator<\/h2>\n<p>A library called Fluent Migrator had most of the required features. You can find out more about <a href=\"https:\/\/github.com\/fluentmigrator\/fluentmigrator\">here<\/a>.<\/p>\n<p>Fluent Migrator is a NuGet package that helps you create migration classes or scripts that can be tracked and automated in code.<\/p>\n<h2>How it Works<\/h2>\n<p>Migrations can be written programmatically, or you can execute SQL scripts directly from code. You can target multiple databases and have control down to the transaction level which can also be helpful.<\/p>\n<p>&nbsp;<\/p>\n<p>Some of the main points include:<\/p>\n<ul>\n<li>Fluent Migrator tracks migrations using a table called VersionInfo<\/li>\n<li>Each record in this table contains a unique VersionId, Date Applied and Description for a single migration<\/li>\n<li>Databases upgrades or rollbacks are handled in unique migrations classes within your solution.<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h2>Creating a Unique Version Id<\/h2>\n<p>A unique version is assigned to each class along with a description using the Migration attribute.\u00a0 The upgrade and downgrade methods are implemented:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"csharp\">[Migration(12345,\"a description\")]\r\npublic class M001_DB_DescriptionOfTask : Migration\r\n{\r\n  public override void Up()\r\n  {\r\n  \u00a0 Execute.Script(Directory.GetCurrentDirectory() + @\"YourUpgradeScript.SQL\");\r\n  }\r\n  public override void Down()\r\n\u00a0 {\r\n\u00a0\u00a0\u00a0 throw new NotImplementedException();\r\n\u00a0 }\r\n}<\/pre>\n<p>You don\u2019t need to run SQL scripts and just like Entity Framework, you can write code-first migrations.<\/p>\n<p>Having to manually create unique version numbers adds extra friction to the development loop.<\/p>\n<p>Luckily, you can create your own custom attribute to enforce a naming convention for each migration you create.<\/p>\n<h2>Custom Attribute<\/h2>\n<p>Here you can see a custom attribute.\u00a0 It accepts several parameters which are used to generate a unique migration version and description:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"csharp\">public class CustomMigrationAttribute : FluentMigrator.MigrationAttribute\r\n{\r\n  public CustomMigrationAttribute(int branchId, int year, int month, int day, int hour, int minute, string author, FluentMigrator.TransactionBehavior transactionBehavior = \r\n                                  FluentMigrator.TransactionBehavior.Default) : base(CalculateValue(branchId, year, month, day, hour, minute), transactionBehavior)\r\n  {\r\n    this.Author = author;\r\n  }\r\n     \r\n  public string Author { get; private set; }\r\n      \r\n  private static long CalculateHash(int branchId, int year, int month, int day, int hour, int minute)\r\n  {\r\n            return branchNumber * 1000000000000L + year * 100000000L + month * 1000000L + day * 10000L + hour * 100L + minute;\r\n  }\r\n}\r\n<\/pre>\n<p>&nbsp;<\/p>\n<p>By default, each migration will run in a database transaction. I\u2019ve extended the above example (taken from Fluent Migrator docs) to include additional parameter FluentMigrator.TransactionBehavior transactionBehavior .<\/p>\n<p>This lets you specify if you want to wrap the C# migration in a transaction (<strong>Default<\/strong>). Setting it to <strong>None<\/strong> will prevent the transaction from being added.<\/p>\n<p>Learn more about custom attributes <a href=\"https:\/\/fluentmigrator.github.io\/articles\/migration\/migration-attribute-custom.html\">here<\/a>:<\/p>\n<h2>SQL Code with Transactions<\/h2>\n<p>As mentioned, Fluent Migrator will wrap each call within a database transaction.\u00a0 This means your migration will fail if you have any SQL scripts with logic such as :<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"sql\">IF (@@TRANCOUNT&gt;0)\r\nBEGIN\r\n  RAISERROR('Transaction Count mismatch',16,1)\r\nEND\r\nEND TRY\r\nBEGIN CATCH\r\n  IF (@@TRANCOUNT&gt;0)\r\n\u00a0 BEGIN\r\n  \u00a0 ROLLBACK\r\n \u00a0\u00a0 SELECT 'Error inserting data schema, Operation rolled Back' AS [MSG]\r\nEND<\/pre>\n<p>&nbsp;<\/p>\n<p>Set <strong>FluentMigrator.TransactionBehavior transactionBehavior = FluentMigrator.TransactionBehavior.None<\/strong> to prevent this from happening.<\/p>\n<h2>Example<\/h2>\n<p>You can see an example migration class here.\u00a0 This class runs an upgrade that adds a column Age to the User table.<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"csharp\">[CustomMigration(0007, 2023, 02, 18, 08, 00, \"Jamie Maguire\")]\r\npublic class M0007_Sandbox_AlterUserAddAge : Migration\r\n{\r\n    public override void Up()\r\n    {\r\n        Execute.Script(Directory.GetCurrentDirectory() + @\"\\..\\..\\..\\..\\SandboxMigrations\\Scripts\\2023\\02\\0007\\AlterUserAddAge.SQL\");\r\n    }\r\n\r\n    public override void Down()\r\n    {\r\n        throw new NotImplementedException();\r\n    }\r\n}\r\n<\/pre>\n<p>&nbsp;<\/p>\n<p>The SQL script is nothing fancy:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"sql\">ALTER TABLE [User]\r\n\r\nADD Age int<\/pre>\n<p>&nbsp;<\/p>\n<p>After a migration is complete, the Version table is updated.\u00a0 This signals this migration has run and can be ignored on subsequent runs:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"1936\" height=\"858\" class=\"alignnone wp-image-3235 size-full\" src=\"https:\/\/jamiemaguire.net\/wp-content\/uploads\/2023\/02\/Pasted.png\" srcset=\"https:\/\/jamiemaguire.net\/wp-content\/uploads\/2023\/02\/Pasted.png 1936w, https:\/\/jamiemaguire.net\/wp-content\/uploads\/2023\/02\/Pasted-750x332.png 750w, https:\/\/jamiemaguire.net\/wp-content\/uploads\/2023\/02\/Pasted-1400x620.png 1400w, https:\/\/jamiemaguire.net\/wp-content\/uploads\/2023\/02\/Pasted-768x340.png 768w, https:\/\/jamiemaguire.net\/wp-content\/uploads\/2023\/02\/Pasted-1536x681.png 1536w, https:\/\/jamiemaguire.net\/wp-content\/uploads\/2023\/02\/Pasted-816x362.png 816w\" sizes=\"auto, (max-width: 1936px) 100vw, 1936px\" \/><\/p>\n<h2>Main Code<\/h2>\n<p>All of this is wrapped in a method that fetches the connection string from a JSON config file and any projects with migrations are loaded in using reflection:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"csharp\">private static void Main(string[] args)\r\n{\r\n    var builder = new ConfigurationBuilder()\r\n                       .SetBasePath(Directory.GetCurrentDirectory())\r\n                       .AddJsonFile(\"appsettings.json\", optional: true, reloadOnChange: true);\r\n\r\n\r\n        string connectionString = string.Empty;\r\n\r\n        connectionString = builder.Build().GetConnectionString(\"Sandbox\");\r\n\r\n        var sc = new ServiceCollection()\r\n        \/\/ Add common FluentMigrator services\r\n        .AddFluentMigratorCore()\r\n        .ConfigureRunner(rb =&gt; rb\r\n        \/\/ Add SQL support to FluentMigrator\r\n        .AddSqlServer()\r\n        \/\/ Set the connection string\r\n        .WithGlobalConnectionString(connectionString)\r\n        \/\/ Define the assembly containing the migrations\r\n        .ScanIn(typeof(SandboxMigrations.Sandbox).Assembly)\r\n               .For.Migrations())\r\n        \/\/ Enable logging to console in the FluentMigrator way\r\n        .AddLogging(lb =&gt; lb.AddFluentMigratorConsole())\r\n        \/\/ Build the service provider\r\n        .BuildServiceProvider(false);\r\n\r\n        using (var sandpitScope = sc.CreateScope())\r\n        {\r\n            UpdateDatabase(sandpitScope.ServiceProvider);\r\n        }\r\n    }\r\nstatic void UpdateDatabase(IServiceProvider serviceProvider)\r\n{\r\n      \/\/ Instantiate the runner\r\n       var runner = serviceProvider.GetRequiredService&lt;IMigrationRunner&gt;();\r\n      \/\/ Execute the migrations\r\n      if (runner.HasMigrationsToApplyUp())\r\n      {\r\n          runner.ListMigrations();\r\n          runner.MigrateUp();\r\n      }\r\n}\r\n<\/pre>\n<h2>Demo<\/h2>\n<p>You can watch the video of it in action <a href=\"https:\/\/www.youtube.com\/watch?v=o9Iwpb2RrsQ\">here<\/a> on YouTube or check it out here:<\/p>\n<div class=\"entry-content-asset videofit\"><iframe loading=\"lazy\" title=\"How To Perform SQL Server Database Updates using Fluent Migrator\" width=\"720\" height=\"405\" src=\"https:\/\/www.youtube.com\/embed\/o9Iwpb2RrsQ?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/div>\n<h2>Tips<\/h2>\n<p>You can tell Fluent Migrator to emit migrations to the console but setting the following command ListMigrations() :<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"1190\" height=\"578\" class=\"wp-image-3236 size-full aligncenter\" src=\"https:\/\/jamiemaguire.net\/wp-content\/uploads\/2023\/02\/Pasted-1.png\" srcset=\"https:\/\/jamiemaguire.net\/wp-content\/uploads\/2023\/02\/Pasted-1.png 1190w, https:\/\/jamiemaguire.net\/wp-content\/uploads\/2023\/02\/Pasted-1-750x364.png 750w, https:\/\/jamiemaguire.net\/wp-content\/uploads\/2023\/02\/Pasted-1-768x373.png 768w, https:\/\/jamiemaguire.net\/wp-content\/uploads\/2023\/02\/Pasted-1-816x396.png 816w\" sizes=\"auto, (max-width: 1190px) 100vw, 1190px\" \/><\/p>\n<p>You can override the the default behaviour of wrapping all migrations in a transaction by changing the configuration options of the Fluent Migrator &#8220;runner&#8221;.<\/p>\n<h2>Visual Studio Solution Structure:<\/h2>\n<p>A solution structure like the following helps you create a logical grouping for each migration in the project\/db:<\/p>\n<p>Solution Name-&gt;<\/p>\n<ul>\n<li>Common (Shared Project, contains the Custom Migration Attribute)<\/li>\n<li>Migrations (Contains migrations for each database)\n<ul>\n<li>Year\n<ul>\n<li>Month<\/li>\n<li>M{TaskId}_Database_Description.cs<\/li>\n<\/ul>\n<\/li>\n<li>Scripts (Contains migration class and any SQL classes)\n<ul>\n<li>Year\n<ul>\n<li>Month<\/li>\n<li>TaskId<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>How this looks in Visual Studio:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"418\" height=\"897\" class=\"wp-image-3237 size-full aligncenter\" src=\"https:\/\/jamiemaguire.net\/wp-content\/uploads\/2023\/02\/Pasted-2.png\" srcset=\"https:\/\/jamiemaguire.net\/wp-content\/uploads\/2023\/02\/Pasted-2.png 418w, https:\/\/jamiemaguire.net\/wp-content\/uploads\/2023\/02\/Pasted-2-349x750.png 349w\" sizes=\"auto, (max-width: 418px) 100vw, 418px\" \/><\/p>\n<p>That\u2019s it!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A while ago I had to implement a solution to automatically update multiple databases in a mature production environment as part of a CI\/CD pipeline.\u00a0 The environment and setup had some nuances. Getting licenses for&#8230;<\/p>\n","protected":false},"author":1,"featured_media":3241,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","footnotes":""},"categories":[6,90,98,99],"tags":[],"class_list":["post-3234","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-c","category-developer-life","category-legacy-code","category-mature-applications"],"_links":{"self":[{"href":"https:\/\/jamiemaguire.net\/index.php\/wp-json\/wp\/v2\/posts\/3234","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jamiemaguire.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jamiemaguire.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jamiemaguire.net\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jamiemaguire.net\/index.php\/wp-json\/wp\/v2\/comments?post=3234"}],"version-history":[{"count":0,"href":"https:\/\/jamiemaguire.net\/index.php\/wp-json\/wp\/v2\/posts\/3234\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/jamiemaguire.net\/index.php\/wp-json\/wp\/v2\/media\/3241"}],"wp:attachment":[{"href":"https:\/\/jamiemaguire.net\/index.php\/wp-json\/wp\/v2\/media?parent=3234"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jamiemaguire.net\/index.php\/wp-json\/wp\/v2\/categories?post=3234"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jamiemaguire.net\/index.php\/wp-json\/wp\/v2\/tags?post=3234"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}