Legacy to Unified alerts migration in Grafana¶

We are using legacy alert system for Grafana in all of our monitoring application. Grafana introduced new unified alerting system from version 9 and marked legacy one deprecated. We have upgraded our Grafana to version 10 already but legacy alerting is still there. From Grafana version 11,legacy alert will be fully removed and we have to migrated to unified one.

Resources¶

• Grafana legacy to unified alert migration documentation here
• Difference and limitations which will be noticed after migration here

Here we will describe whole migration step required for Grafana production in DT Growth project.

• Create proper dashboard folder structure : Read and write access to legacy dashboard alerts and Grafana alerts are governed by the permissions of the folders storing them. So we have to organise dashboard folder with appropriate structure and permissions. If the dashboards are not correctly setup with folder and permission in the legacy one, we will see some random dashboard folder along with random evaluation group(for unified alerting we have to select folder and create evaluation group which is not available in legacy one) created after migration done and it will cause empty dashboard or empty evaluation group alerts. We can easily fix it by arranging folder with inherited permission.

Now you will see difference between old and new grafana interface for Growth.

Folder structure in Legacy system

Folder structure should be in new system

Dashboard permission in old system

Dashboard permission should be in new system 

We have to remove any user which is not inherited by folder otherwise migration will create two folder structure with separate evaluation group which will create lots of extra unwanted alerts. So for proper migration we have remove extra user and after migration we can add removed user again. - Fixed old panel wizard :In some dashboard we are using old panel wizard which doesn't show any data as this wizard already invalid. We need to set appropriate wizard before new migration. Below you will see an example:

- New alert wizard :In old system alerts only we have interval and conditions setup based on query tab but in new system you will see Query , conditions and all other new features in the same tab of alerts where you can configure set independent configuration per query or set query level conditions as well.

In old system: 

In new System: 

- *No data and Error handling setup:NoData and Error handling options will create a lots of alerts if we can not appropriately configure it in new alert system. In legacy we have options for Nodata and Error Keep Last State*, which will be replaced in new alert system as in new system its not available.

• In NoData: Keep Last State => Nodata

• In Error: Keep Last State => ErrorSo we need to configure this No data and Error handlingin a correct way to avoid any noData or Error related alerts after migration.

  Lets see the difference in old and new system 

  Old system:

  

  

  In new system:

  

  

• Handle paused alerts:In the legacy alerts system we have an option to paused alerts but in new system we don't have it rather we can use silence option for this. After the migration happen all paused alerts activated again as there is no action is taken by migration job to make paused alerts to silence automatically.

In old system we can filter paused alerts:

In the new system we don't have this options but we have silence option.

####

Before migration we have to consider above points to be performed properly to successfully migrate legacy to unified alerting.