fill missing dates in mysql

How to Fill Missing Dates in MySQL?

Ubiq

Many times, we need to fill missing dates in MySQL data, by filling missing rows for those dates. Here’s how to fill in missing dates and date sequence gaps to create a proper sales report that doesn’t have any missing rows.

 

How to Fill Missing Dates in MySQL?

Here are the steps to fill missing dates in MySQL. Let’s say you have the following table sales(order_date, sale)

mysql> create table sales(order_date date,sale int);

mysql> insert into sales(order_date,sale) values('2020-04-01',212),
     ('2020-04-04',220),
     ('2020-04-05',120),
     ('2020-04-07',200),
     ('2020-04-08',222),
     ('2020-04-10',312),
     ('2020-04-11',225),
     ('2020-04-12',212);

mysql> select * from sales;
+------------+------+
| order_date | sale |
+------------+------+
| 2020-04-01 |  212 |
| 2020-04-04 |  220 |
| 2020-04-05 |  120 |
| 2020-04-07 |  200 |
| 2020-04-08 |  222 |
| 2020-04-10 |  312 |
| 2020-04-11 |  225 |
| 2020-04-12 |  212 |
+------------+------+

As you can see in table above, there is no data for many dates in between such as April 2, 3, etc.

If we try to plot this data on a graph, it will look something like

fill missing dates in mysql

Such a graph can be misleading as it doesn’t show column gaps for missing dates.

Bonus Read: SQL Query to Compare Product Sales By Month

 

In order to fill missing date rows in MySQL, we will need to create a helper table that contains all required dates, without any sequence gaps or missing dates and then join it with our sales table.

Here’s how to create our helper table calendar.

mysql> CREATE TABLE calendar (datefield DATE);

mysql> insert into calendar(datefield) values('2020-04-01'),
     ('2020-04-02'),
     ('2020-04-03'),
     ('2020-04-04'),
     ('2020-04-05'),
     ('2020-04-06'),
     ('2020-04-07'),
     ('2020-04-08'),
     ('2020-04-09'),
     ('2020-04-10'),
     ('2020-04-11'),
     ('2020-04-12'),
     ('2020-04-13'),
     ('2020-04-14'),
     ('2020-04-15'),
     ('2020-04-16'),
     ('2020-04-17'),
     ('2020-04-18'),
     ('2020-04-19'),
     ('2020-04-20'),
     ('2020-04-21'),
     ('2020-04-22'),
     ('2020-04-23'),
     ('2020-04-24'),
     ('2020-04-25'),
     ('2020-04-26'),
     ('2020-04-27'),
     ('2020-04-28'),
     ('2020-04-29'),
     ('2020-04-30');

mysql> select * from calendar;
+------------+
| datefield  |
+------------+
| 2020-04-01 |
| 2020-04-02 |
| 2020-04-03 |
| 2020-04-04 |
| 2020-04-05 |
| 2020-04-06 |
| 2020-04-07 |
| 2020-04-08 |
| 2020-04-09 |
| 2020-04-10 |
| 2020-04-11 |
| 2020-04-12 |
| 2020-04-13 |
| 2020-04-14 |
| 2020-04-15 |
| 2020-04-16 |
| 2020-04-17 |
| 2020-04-18 |
| 2020-04-19 |
| 2020-04-20 |
| 2020-04-21 |
| 2020-04-22 |
| 2020-04-23 |
| 2020-04-24 |
| 2020-04-25 |
| 2020-04-26 |
| 2020-04-27 |
| 2020-04-28 |
| 2020-04-29 |
| 2020-04-30 |
+------------+

There is also a shorter way to create calendar table, if you are comfortable with using stored procedures.

mysql> DELIMITER |
     CREATE PROCEDURE fill_calendar(start_date DATE, end_date DATE)
     BEGIN
       DECLARE crt_date DATE;
       SET crt_date=start_date;
       WHILE crt_date <= end_date DO INSERT INTO calendar VALUES(crt_date); 
    SET crt_date = ADDDATE(crt_date, INTERVAL 1 DAY); 
    END WHILE; END | 
    DELIMITER ;

You can call this function using the CALL function. For date range from April 1 to 30,

mysql> CALL fill_calendar('2020-04-01', '2020-04-31');

Bonus Read: How to Calculate Sales Per Month in MySQL

 

Finally, we do a LEFT join of calendar table with sales to fill missing date rows in MySQL.

mysql> select datefield,sum(sale) from calendar 
left join sales on datefield=order_date 
group by datefield;
+------------+-----------+
| datefield  | sum(sale) |
+------------+-----------+
| 2020-04-01 |       212 |
| 2020-04-02 |      NULL |
| 2020-04-03 |      NULL |
| 2020-04-04 |       220 |
| 2020-04-05 |       120 |
| 2020-04-06 |      NULL |
| 2020-04-07 |       200 |
| 2020-04-08 |       222 |
| 2020-04-09 |      NULL |
| 2020-04-10 |       312 |
| 2020-04-11 |       225 |
| 2020-04-12 |       212 |
| 2020-04-13 |      NULL |
| 2020-04-14 |      NULL |
| 2020-04-15 |      NULL |
| 2020-04-16 |      NULL |
| 2020-04-17 |      NULL |
| 2020-04-18 |      NULL |
| 2020-04-19 |      NULL |
| 2020-04-20 |      NULL |
| 2020-04-21 |      NULL |
| 2020-04-22 |      NULL |
| 2020-04-23 |      NULL |
| 2020-04-24 |      NULL |
| 2020-04-25 |      NULL |
| 2020-04-26 |      NULL |
| 2020-04-27 |      NULL |
| 2020-04-28 |      NULL |
| 2020-04-29 |      NULL |
| 2020-04-30 |      NULL |
+------------+-----------+

As you can see, the above table contains all dates, with sales for those rows where data is present and NULL for missing date rows.

If you want to replace these NULLs with 0, you can use an IFNULL function,

mysql> select datefield,IFNULL(sum(sale),0) 
from calendar left join sales on datefield=order_date 
group by datefield;
+------------+---------------------+
| datefield  | IFNULL(sum(sale),0) |
+------------+---------------------+
| 2020-04-01 |                 212 |
| 2020-04-02 |                   0 |
| 2020-04-03 |                   0 |
| 2020-04-04 |                 220 |
| 2020-04-05 |                 120 |
| 2020-04-06 |                   0 |
| 2020-04-07 |                 200 |
| 2020-04-08 |                 222 |
| 2020-04-09 |                   0 |
| 2020-04-10 |                 312 |
| 2020-04-11 |                 225 |
| 2020-04-12 |                 212 |
| 2020-04-13 |                   0 |
| 2020-04-14 |                   0 |
| 2020-04-15 |                   0 |
| 2020-04-16 |                   0 |
| 2020-04-17 |                   0 |
| 2020-04-18 |                   0 |
| 2020-04-19 |                   0 |
| 2020-04-20 |                   0 |
| 2020-04-21 |                   0 |
| 2020-04-22 |                   0 |
| 2020-04-23 |                   0 |
| 2020-04-24 |                   0 |
| 2020-04-25 |                   0 |
| 2020-04-26 |                   0 |
| 2020-04-27 |                   0 |
| 2020-04-28 |                   0 |
| 2020-04-29 |                   0 |
| 2020-04-30 |                   0 |
+------------+---------------------+

Bonus Read : How to Create Dynamic Pivot Table in MySQL

 

When we fill missing dates in MySQL, we will also get data for future dates. If you also want to remove these extra 0 rows that are present in calendar after the latest order_date you can do so with WHERE clause

mysql> select datefield,IFNULL(sum(sale),0) 
from calendar left join sales on datefield=order_date 
where datefield 
between (SELECT MIN(DATE(order_date)) FROM sales) 
AND
 (SELECT MAX(DATE(order_date)) FROM sales) 
group by datefield;
+------------+---------------------+
| datefield  | IFNULL(sum(sale),0) |
+------------+---------------------+
| 2020-04-01 |                 212 |
| 2020-04-02 |                   0 |
| 2020-04-03 |                   0 |
| 2020-04-04 |                 220 |
| 2020-04-05 |                 120 |
| 2020-04-06 |                   0 |
| 2020-04-07 |                 200 |
| 2020-04-08 |                 222 |
| 2020-04-09 |                   0 |
| 2020-04-10 |                 312 |
| 2020-04-11 |                 225 |
| 2020-04-12 |                 212 |
+------------+---------------------+

After you fill missing dates in MySQL, you can use a reporting tool to plot this data on a bar chart or dashboard and share it with your team. Here’s an example of a bar chart that shows daily sales, created using Ubiq.

fill date gaps in mysql

See the difference in 2 charts!

If you want to create charts, dashboards & reports from MySQL database, you can try Ubiq. We offer a 14-day free trial.

Related posts:

How To Create Index in MySQL
Common Table Expression in MySQL
MySQL CAST - How to Type Cast in MySQL
mm

About Ubiq

Ubiq is a powerful dashboard & reporting platform. Build dashboards, charts & reports for your business in minutes. Try it for free!