Ever wondered why the MySQL command line tool don’t have any sort of progress information or status update? Well, you’re not alone if you do. Especially importing a GB sized dump file can take a long time, especially if the table engine was InnoDB.

There are some clever utilities which can be used to accomplish just that, by giving the user simple, yet useful information about the progress of their process. Let’s look at them.

Bar (Command Line Progress Bar)

The Bar utility, or in full terms Command Line Progress Bar can be downloaded from Sourceforge. If you are running Ubuntu, it’s as simple as running ‘sudo apt-get install bar’ to install it. You then simply pipe your MySQL import and get a nice status bar.

shell> bar -if=mysql_db_data.sql | mysql

Piper Viewer

To get a bit fancier, Piper Viewer can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion. Observe multiple instances working in tandem,  a visual indicator of relative throughput in a complex pipeline. Download pv here.

pv -cN gzip mysql_db_data.sql.gz | gzip -d | pv -cN mysql | mysql

Either utility will undoubtedly shed more light into your restoration processes, and let you know whether the process is still running if in doubt and wondering after an hour or more.

I’ve been asked many times how to delete records in multiple tables in MySQL. Here are couple of methods that should get anyone started.

Simple Inner Join Method

The simple inner join method uses commas with an implicit inner join like in the example below. Here we’re using an items and price table, where each table has an itemId field which is what we join them on.

DELETE i.*, p.*
FROM items i, price p
WHERE i.itemId = p.itemId
AND i.itemId = 101

Traditional Inner Join Method

In the traditional inner join we’re keeping the join statement clear of the where clause, but it works as well. Whatever personal preference you may have. Note that you can delete individual table records from one table only, if you require to do so.

DELETE i.*, p.*
FROM items i
INNER JOIN price p
ON i.itemId = p.itemId
WHERE i.itemId = 101

Delete records that are missing on the second table

Using a left join, we can also delete records that have no corresponding record in the joined table. The following query accomplishes that. In the example below, we delete all records in the items table that have no record in the price table.

DELETE i.*
FROM items i
LEFT JOIN price p
ON i.itemId = p.itemId
WHERE p.itemId IS NULL

I don’t remember how many times I was asked about an equivalent term of the infamous “NOLOCK” hint for mysql database server, hence I thought it was worth to write about it here. “WITH (NOLOCK)” is a transaction isolation levels that defines how data is available during an update, or with other words it is a property that defines at what point changes made by an update operation will become available in a row, table or database to other processes.

The official SQL standard defines four isolation levels:

READ COMMITTED
READ UNCOMMITTED
REPEATABLE READ
SERIALIZABLE

Oracle, SQL Server and MySQL support isolation levels. During an operation, the database engine places certain locks to maintain data integrity. Different types of locking apply to different databases (Oracle vs. MySQL), or table types (eg. MyISAM vs. InnoDB).

When WITH (NOLOCK) is used with SQL Server, the statement does not place a lock nor honor exclusive locks on table. The MySQL equivalent is READ UNCOMMITTED, also known as “dirty read” because it is the lowest level of isolation. If we specify a table hint then it will override the current default isolation level. MySQL default isolation level is REPEATABLE READ which means locks will be placed for each operation, but multiple connections can read data concurrently.

SQL Server WITH (NOLOCK) looks like this:

SELECT * FROM TABLE WITH (nolock)

To achieve the same with MySQL, we change the session isolation mode using the SET SESSION command.

SET SESSION TRANSACTION ISOLATION LEVEL READ UNCOMMITTED ;
SELECT * FROM TABLE_NAME ;
SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ ;

This statement will work similar to WITH (NOLOCK) i.e READ UNCOMMITTED data. We can also set the isolation level for all connections globally:

SET GLOBAL TRANSACTION ISOLATION LEVEL READ UNCOMMITTED ;

In addition, two system variables related to isolation also level exist in MySQL server:

SELECT @@global.tx_isolation; (global isolation level)
SELECT @@tx_isolation; (session isolation level)

Or set the isolation level inside a transaction:

SET TRANSACTION ISOLATION LEVEL REPEATABLE READ
GO

I’ve been asked the question “How can I return duplicate rows only from a MySQL db table” so many times already, that I’ve decided to post it here in a short article.

It is not something intuitive or readily available (at least it seems), but the solution is short and very simple.

While this query:

SELECT DISTINCT column1
FROM table1

gives us all records without the duplicates, this one returns only the duplicate ones:

SELECT DISTINCT column1
FROM table1
GROUP BY column1
HAVING COUNT(column1) > 1

And by increasing the having count, you can retrieve records with multiple occurrences.

Ever come across a situation, where you’d like to check all tables in a database and have them all repaired and optimized? My guess is yes.

In case you didn’t know, there is a helpful MySQL utility called mysqlcheck, available as of version 3.23.38. It does exactly what we need.

To check all tables in all databases for corruption and errors and also fix them in one go, this is your command:

mysqlcheck -u username -p password  –check –optimize –auto-repair –all-databases

mysqlcheck executes statements like CHECK TABLE, REPAIR TABLE, ANALYZE TABLE, and OPTIMIZE TABLE and chooses the best statements for any given operation and storage engine.

Note that the operations complete a lot faster if you can afford to to disable any external services, especially if your database is large.

These instructions lead you thru the installation of the latest LAMP Stack on MacOS X Snow Leopard.

1 – Download the installation image from MySQL website here. Then double-click to mount and open the disk image.

2 – Install MySQL Server by double clicking the package “mysql-5.1.*****.pkg” and follow the menu, accepting the default values, unless you want to change something and know exactly what you’re doing.

3 – Install MySQL Startup Item by double-clicking the package “MySQLStartupitem.pkg” and follow the menu.

4 – Install MySQL Preference Pane by double-clicking the file “MySQL.prefPane” and follow the menu. This item will simplify the management of your SQL Server. You can now use the “System Preferences” panel to start and stop the database server.

5 – Enable the php module in your apache config file. You might know that Snow Leopard already ships with Apache 2.2 and PHP 5.3, but it needs a couple of tweaks to make it work smoothly. So, open /etc/apache2/httpd.conf and search for “php5_module”. Remove the comment (#) in front of the line, save and close the file, then restart apache (sudo apachectl restart)

6 – Testing Apache/PHP. Let’s create an info file so we know what we got. Navigate to /Library/WebServer/Documents and create a file called phpinfo.php. Put the following line into the file, save and close. “<?php phpinfo(); ?>”. Now open your favorite browser and hit it (http://localhost/phpinfo.php). You should see a page like the one below, which shows you configuration details, modules enabled and other useful information.

7 – There is no php.ini out of the box (as you may have discovered by looking at the phpinfo page), but there is a sample file called “php.ini.default” which we’re going to use. Copy this file and save it as php.ini.

8 – Set the default time zone. As PHP 5.3 requires the timezone to be set, let’s do this inside php.ini, so we don’t have to worry about it anymore. You can always override it in your application’s bootstrap, if needed. Open /etc/php.ini and search for “[Date]“. Uncomment the line “date.timezone = ” and add your timezone. Check the php manual for acceptable values. (example: date.timezone = ‘America/Los Angeles’)

And that’s it. You should now have a fully functional LAMP stack with Apache, PHP and MySQL server.

1. Configure the Master

We will need to modify a file called my.cnf, which is the main configuration file for mysql. On most systems it’s located in /etc/ or /etc/mysql/ and it contains all important configuration data.

First, let’s ensure that networking is enabled and mysql listens on all, or at least the client’s IP addresses. We also have to tell mysql what file to write the logs to and from which databases to keep logging, so that the Slave can pick up the changes. And finally we need to assign a unique ID to the Master.

All this info is contained within the following lines in your my.cnf file. Please note that the position of these entries can be spread throughout the file, so you might have to search for each of them.

Restart the server. Then log into mysql and create a user with replication privileges:

The next 3 steps is to lock all tables on the database, take a backup and get the replication sequence ID. We’ll use the backup later on the Slave to establish the baseline, and tell it to start replication starting from the sequence ID.

If you’re performing this action on a high volume production system, I’d recommend to plan these next steps carefully, as the db will be locked for writing during the backup process.

Write down the “Position” number, it’s the sequence ID and the bin log file. Since the tables are locked, you’ll need to open a second terminal and log into mysql and perform the backup:

Once the backup is finished, unlock the tables in the first terminal.

Whew, we’re done with the Master. Wasn’t that hard, was it? Now on to the Slave.

2. Configure the Slave

You’ll have to copy the backup file to the Slave server and extract it (gzip -d). On the Slave, let’s first create the database and restore the dump we took from the Master.

After loging into mysql from the terminal:

Then we’ll go on to configure my.cnf on the Slave. As I stated earlier, the position of these entries can be spread throughout the file, so you might have to search for each of them or they might not even be present, so you have to add them.

3. setup parameters and start replication

Restart MySQL, login to mysql and stop the Slave. You now will need the info you wrote down during the “SHOW MASTER STATUS” command.

Example: CHANGE MASTER TO MASTER_HOST=’192.168.1.1′, MASTER_USER=’slave_user’, MASTER_PASSWORD=’1234′, MASTER_LOG_FILE=’mysql-bin.001′, MASTER_LOG_POS=256;

Finally, we start the Slave with the new settings and get replication started. Still from the terminal, logged in to mysql:

That’s all there is to it. There are commands like “SHOW SLAVE STATUS” that let’s you check if replication is working.

Update:

If you have problems getting “Slave_IO_Running” to “Yes” state, you might have to flush the tables on the slave, like this:

STOP SLAVE;

FLUSH TABLES WITH READ LOCK;

UNLOCK TABLES;

START SLAVE;

INSERT IGNORE is the syntax that does mimic INSERT IF NOT EXISTS, as there is no direct command as stated in the title, at least not in the current release of MySql.
The statement INSERT IGNORE (and to some extent REPLACE INTO) does essentially the same thing, inserting a record if that given record does not exists.
See the following samples:

Sample:

INSERT IGNORE INTO `table`
SET ‘column_1` = `value_1`,
`column_2` = `value_2′;

If you’re concerned with existing data that might result in duplicate records, or error out due to duplicate id’s, use REPLACE. MySql will check if the record exists, replace the values given and insert it if the record does not exist.

Sample:

REPLACE INTO `table`
SET ‘column_1` = `value_1`,
`column_2` = `value_2′;

References: INSERT IGNORE and REPLACE INTO