MySQL insert into select triggers lock table

HelloCoder-HaC 2020-11-10 09:13:45
mysql insert select triggers lock


On Friday, HaC I'm going online , There is a script to execute , A table needs to be backed up before execution .

O & M boss :“ Why does the backup of this table take so long ,,??”

1 Seconds passed ......2 Seconds passed ......

During the operation feedback system, a large number of orders were overtime .

Big guy found me , ask :“ How do you back up ?”

I :“insert into select * from ah !”

bosses :“?? You don't want to get mixed up ?”

It's another day that I'm hated by the boss , In order not to pack up and leave , I decided to learn some common methods of table backup .

MySQL In general, we usually use These two methods :

  1. INSERT INTO SELECT
  2. CREATE TABLE AS SELECT

notes : This article is only for MySQL innodb engine , Transactions are repeatable RR, The database version is 5.5

1.INSERT INTO SELECT

insert into Table2(field1,field2,...) select value1,value2,... from Table1

Be careful

(1) Request target table Table2 There must be , And the fields field,field2... There must also be

(2) Be careful Table2 Primary key constraint for , If Table2 It has a primary key and is not empty , be field1, field2... The primary key must be included in

When executing a statement ,MySQL It's line by line locking ( Scan one lock one ), Until all eligible data is locked , Release the lock after execution . So when the business is going on , Don't use this method .

stay RR Under isolation level , Also add row lock and clearance lock

Give me a chestnut :

CREATE TABLE `t` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`c` int(11) DEFAULT NULL,
`d` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `c` (`c`)
) ENGINE=InnoDB;
insert into t values(null, 1,1);
insert into t values(null, 2,2);
insert into t values(null, 3,3);
insert into t values(null, 4,4);
create table t2 like t;

perform

begin;
insert into t2(c,d) select c,d from t;

First not commit; This statement is against the table t The primary key index has been added with (-∞,1] This next-key lock

Open a new one Navicat window , Simulate a new transaction entering , At this time, execute the following sentence sql We need to wait

insert into t values(-1,-1,-1);

 lock

It really locks the watch ~ I can't write it in , I finally know why the order is over time .

Back the pot, back the pot .

If you really want to use INSERT INTO SELECT This method , You can use the following method to optimize :

  1. Conditional , Force Index , Don't scan the whole table , for example
INSERT INTO Table2 SELECT
*
FROM
Table1 FORCE INDEX (create_time)
WHERE
update_time <= '2020-03-08 00:00:00';
  1. add limit 100,100 such , Limited quantity

2. CREATE TABLE AS SELECT

create table as select Will create a table that doesn't exist , It can also be used to copy a table .

1. create table t3 as select * from t where 1=2;
-- Create a table structure with t As like as two peas , Copy only structures, not data ;
2.create table t3 as select * from t ;
-- Create a table structure with t As like as two peas , Replication structures also replicate data ;( Indexes don't create )
3.create table t3(`id`,`a`) as select `id`,`c` from t;
-- Create a table structure with t As like as two peas , Replication structures also replicate data , But specify the column name of the new table ;

The last two formats , If you follow the appropriate query criteria , You can only copy the data that meets the criteria to a new table . such as :

create table table1 as select * from table2 where columns1>=1;

Table replication for large tables and multi fields , Consider whether each field is required , If it's not necessary , Can I customize the selection field , This will greatly increase the copy time .

CREATE table table1 as SELECT id FROM table2; -- Just copy id This column 

Be careful This table creation process locks the table throughout the process . Statement execution finished , To release Metadata lock .

MDL Its full name is metadata lock, Metadata lock .MDL Lock is mainly used to maintain data consistency of table metadata , There are activities on the list ( Explicit or implicit ) When , You cannot write metadata . So from MySQL5.5 Version started to be introduced MDL lock , To protect the metadata information of the table , To solve or guarantee DDL Operation and DML Consistency between operations .

Be careful :

  1. The new table does not automatically create the same index as the original table .( That is, the index of the replicated table will disappear )

3 . difference

  • First , The biggest difference is that they belong to different types of statements ,INSERT INTO SELECT yes DML sentence ( Data operation language ,SQL Processing data and other operations are collectively referred to as data manipulation language ), After completion, it needs to be submitted to take effect ,CREATE TABLE AS SELECT yes DDL sentence ( Data definition language , Used to define and manage SQL The language of all objects in the database ), It takes effect immediately after the execution , No rollback is provided , High efficiency .

  • secondly , Different functions ,INSERT INTO SELECT Just insert data , You have to build a watch first ;CREATE TABLE AS SELECT Then create the table and insert data together .

  • It is not recommended to use when there is a large amount of data Insert into as, Because the insertion efficiency of this statement is very slow .

4. summary

The above is true for replicated tables , It's not a good choice , Share some common methods :

  1. Export to excel, Then spell it sql become insert into values(),(),() In the form of .

  2. Timing task , The logic of the task is to query 100 Bar record , Then multiple threads are divided into several tasks to execute , Like a thread , Every thread 10 Bar record , After inserting , Looking for new 100 Record processing .

  3. mysqldumb Method , for example

    mysqldump -h$host -P$port -u$user --add-locks=0 --no-create-info --single-transaction --set-gtid-purged=OFF db1 t --where="a>900" --result-file=/client_tmp/t.sql
    
  4. export CSV file

select * from db1.t where a>900 into outfile '/server_tmp/t.csv';

The first 3、4 Two methods are suitable for whole table export .

​ 5. When there is little business ( Late at night or something ) Next , have access to create table as select .


Knowledge has increased again .

welfare :

Java Learning materials

版权声明
本文为[HelloCoder-HaC]所创,转载请带上原文链接,感谢

  1. 【计算机网络 12(1),尚学堂马士兵Java视频教程
  2. 【程序猿历程,史上最全的Java面试题集锦在这里
  3. 【程序猿历程(1),Javaweb视频教程百度云
  4. Notes on MySQL 45 lectures (1-7)
  5. [computer network 12 (1), Shang Xuetang Ma soldier java video tutorial
  6. The most complete collection of Java interview questions in history is here
  7. [process of program ape (1), JavaWeb video tutorial, baidu cloud
  8. Notes on MySQL 45 lectures (1-7)
  9. 精进 Spring Boot 03:Spring Boot 的配置文件和配置管理,以及用三种方式读取配置文件
  10. Refined spring boot 03: spring boot configuration files and configuration management, and reading configuration files in three ways
  11. 精进 Spring Boot 03:Spring Boot 的配置文件和配置管理,以及用三种方式读取配置文件
  12. Refined spring boot 03: spring boot configuration files and configuration management, and reading configuration files in three ways
  13. 【递归,Java传智播客笔记
  14. [recursion, Java intelligence podcast notes
  15. [adhere to painting for 386 days] the beginning of spring of 24 solar terms
  16. K8S系列第八篇(Service、EndPoints以及高可用kubeadm部署)
  17. K8s Series Part 8 (service, endpoints and high availability kubeadm deployment)
  18. 【重识 HTML (3),350道Java面试真题分享
  19. 【重识 HTML (2),Java并发编程必会的多线程你竟然还不会
  20. 【重识 HTML (1),二本Java小菜鸟4面字节跳动被秒成渣渣
  21. [re recognize HTML (3) and share 350 real Java interview questions
  22. [re recognize HTML (2). Multithreading is a must for Java Concurrent Programming. How dare you not
  23. [re recognize HTML (1), two Java rookies' 4-sided bytes beat and become slag in seconds
  24. 造轮子系列之RPC 1:如何从零开始开发RPC框架
  25. RPC 1: how to develop RPC framework from scratch
  26. 造轮子系列之RPC 1:如何从零开始开发RPC框架
  27. RPC 1: how to develop RPC framework from scratch
  28. 一次性捋清楚吧,对乱糟糟的,Spring事务扩展机制
  29. 一文彻底弄懂如何选择抽象类还是接口,连续四年百度Java岗必问面试题
  30. Redis常用命令
  31. 一双拖鞋引发的血案,狂神说Java系列笔记
  32. 一、mysql基础安装
  33. 一位程序员的独白:尽管我一生坎坷,Java框架面试基础
  34. Clear it all at once. For the messy, spring transaction extension mechanism
  35. A thorough understanding of how to choose abstract classes or interfaces, baidu Java post must ask interview questions for four consecutive years
  36. Redis common commands
  37. A pair of slippers triggered the murder, crazy God said java series notes
  38. 1、 MySQL basic installation
  39. Monologue of a programmer: despite my ups and downs in my life, Java framework is the foundation of interview
  40. 【大厂面试】三面三问Spring循环依赖,请一定要把这篇看完(建议收藏)
  41. 一线互联网企业中,springboot入门项目
  42. 一篇文带你入门SSM框架Spring开发,帮你快速拿Offer
  43. 【面试资料】Java全集、微服务、大数据、数据结构与算法、机器学习知识最全总结,283页pdf
  44. 【leetcode刷题】24.数组中重复的数字——Java版
  45. 【leetcode刷题】23.对称二叉树——Java版
  46. 【leetcode刷题】22.二叉树的中序遍历——Java版
  47. 【leetcode刷题】21.三数之和——Java版
  48. 【leetcode刷题】20.最长回文子串——Java版
  49. 【leetcode刷题】19.回文链表——Java版
  50. 【leetcode刷题】18.反转链表——Java版
  51. 【leetcode刷题】17.相交链表——Java&python版
  52. 【leetcode刷题】16.环形链表——Java版
  53. 【leetcode刷题】15.汉明距离——Java版
  54. 【leetcode刷题】14.找到所有数组中消失的数字——Java版
  55. 【leetcode刷题】13.比特位计数——Java版
  56. oracle控制用户权限命令
  57. 三年Java开发,继阿里,鲁班二期Java架构师
  58. Oracle必须要启动的服务
  59. 万字长文!深入剖析HashMap,Java基础笔试题大全带答案
  60. 一问Kafka就心慌?我却凭着这份,图灵学院vip课程百度云