Skip to content

Commit 03d3a75

Browse files
macbremacbre
committed
Update README
1 parent 2f2d713 commit 03d3a75

File tree

1 file changed

+76
-76
lines changed

1 file changed

+76
-76
lines changed

‎README.md

Lines changed: 76 additions & 76 deletions
Original file line numberDiff line numberDiff line change
@@ -76,9 +76,84 @@ Examples:
7676
Visit <https://github.com/macbre/index-digest>
7777
```
7878

79-
## An example
79+
## SQL query log
80+
81+
It's a text file with a single SQL query in each line (no line breaks are allowed). Lines that do start with `--` (SQL comment) are ignored. The file can be [generated using `query-digest` when `--sql-log` output mode is selected](https://github.com/macbre/query-digest#output-modes).
82+
83+
An example:
8084

85+
```sql
86+
-- A comment
87+
select * from 0002_not_used_indices order by id
88+
select * from 0002_not_used_indices where foo = 'foo' and id = 2
89+
select count(*) from 0002_not_used_indices where foo = 'foo'
90+
/* foo bar */ select * from 0002_not_used_indices where bar = 'foo'
91+
INSERT IGNORE INTO `0070_insert_ignore` VALUES ('123', 9, '2017-01-01');
8192
```
93+
94+
## Formatters
95+
96+
`index-digest` can return results in various formats (use `--format` to choose one).
97+
98+
### plain
99+
100+
Emits human-readable report to a console. You can disable colored and bold text by setting env variable `ANSI_COLORS_DISABLED=1`.
101+
102+
### syslog
103+
104+
Pushes JSON-formatted messages via syslog, so they can be aggregated using ELK stack.
105+
Use `SYSLOG_IDENT` env variable to customize syslog's `ident` messages are sent with (defaults to `index-digest`).
106+
107+
```
108+
Dec 28 15:59:58 debian index-digest[17485]: {"meta": {"version": "index-digest v0.1.0", "database_name": "index_digest", "database_host": "debian", "database_version": "MySQL v5.7.20"}, "report": {"type": "redundant_indices", "table": "0004_id_foo", "message": "\"idx\" index can be removed as redundant (covered by \"PRIMARY\")", "context": {"redundant": "UNIQUE KEY idx (id, foo)", "covered_by": "PRIMARY KEY (id, foo)", "schema": "CREATE TABLE `0004_id_foo` (\n `id` int(9) NOT NULL AUTO_INCREMENT,\n `foo` varbinary(16) NOT NULL DEFAULT '',\n PRIMARY KEY (`id`,`foo`),\n UNIQUE KEY `idx` (`id`,`foo`)\n) ENGINE=InnoDB DEFAULT CHARSET=latin1", "table_data_size_mb": 0.015625, "table_index_size_mb": 0.015625}}}
109+
```
110+
111+
### yaml
112+
113+
Outputs YML file with results and metadata.
114+
115+
## Checks
116+
117+
You can select which checks should be reported by the tool by using `--checks` command line option. Certain checks can also be skipped via `--skip-checks` option. Refer to `index_digest --help` for examples.
118+
119+
> **Number of checks**: 22
120+
121+
* `redundant_indices`: reports indices that are redundant and covered by other
122+
* `non_utf_columns`: reports text columns that have characters encoding set to `latin1` (utf is the way to go)
123+
* `missing_primary_index`: reports tables with no primary or unique key (see [MySQL bug #76252](https://bugs.mysql.com/bug.php?id=76252) and [Wikia/app#9863](https://github.com/Wikia/app/pull/9863))
124+
* `test_tables`: reports tables that seem to be test leftovers (e.g. `some_guy_test_table`)
125+
* `single_column`: reports tables with just a single column
126+
* `empty_tables`: reports tables with no rows
127+
* `generic_primary_key`: reports tables with [a primary key on `id` column](https://github.com/jarulraj/sqlcheck/blob/master/docs/logical/1004.md) (a more meaningful name should be used)
128+
* `use_innodb`: reports table using storage engines different than `InnoDB` (a default for MySQL 5.5+ and MariaDB 10.2+)
129+
130+
### Additional checks performed on SQL log
131+
132+
> You need to provide SQL log file via `--sql-log` option
133+
134+
* `not_used_columns`: checks which columns were not used by SELECT queries
135+
* `not_used_indices`: checks which indices are not used by SELECT queries
136+
* `not_used_tables`: checks which tables are not used by SELECT queries
137+
* `queries_not_using_index`: reports SELECT queries that do not use any index
138+
* `queries_using_filesort`: reports SELECT queries that require filesort ([a sort can’t be performed from an index and quicksort is used](https://www.percona.com/blog/2009/03/05/what-does-using-filesort-mean-in-mysql/))
139+
* `queries_using_temporary`: reports SELECT queries that require a temporary table to hold the result
140+
* `queries_using_full_table_scan`: reports SELECT queries that require a [full table scan](https://dev.mysql.com/doc/refman/5.7/en/table-scan-avoidance.html)
141+
* `selects_with_like`: reports SELECT queries that use `LIKE '%foo'` conditions (they can not use an index)
142+
* `insert_ignore`: reports [queries using `INSERT IGNORE`](https://medium.com/legacy-systems-diary/things-to-avoid-episode-1-insert-ignore-535b4c24406b)
143+
* `select_star`: reports [queries using `SELECT *`](https://github.com/jarulraj/sqlcheck/blob/master/docs/query/3001.md)
144+
* `having_clause`: reports [queries using `HAVING` clause](https://github.com/jarulraj/sqlcheck/blob/master/docs/query/3012.md)
145+
* `high_offset_selects`: report [SELECT queries using high OFFSET](https://www.percona.com/blog/2008/09/24/four-ways-to-optimize-paginated-displays/)
146+
147+
### Additional checks performed on tables data
148+
149+
> You need to use `--analyze-data` command line switch. Please note that these checks will query your tables. **These checks can take a while if queried columns are not indexed**.
150+
151+
* `data_too_old`: reports tables that have really old data, maybe it's worth checking if such long data retention is actually needed (**defaults to three months threshold**, can be customized via `INDEX_DIGEST_DATA_TOO_OLD_THRESHOLD_DAYS` env variable)
152+
* `data_not_updated_recently`: reports tables that were not updated recently, check if it should be up-to-date (**defaults a month threshold**, can be customized via `INDEX_DIGEST_DATA_NOT_UPDATED_RECENTLY_THRESHOLD_DAYS` env variable)
153+
154+
## An example report
155+
156+
```sql
82157
$ index_digest mysql://index_digest:qwerty@localhost/index_digest --sql-log sql/0002-not-used-indices-log
83158
------------------------------------------------------------
84159
Found 85 issue(s) to report for "index_digest" database
@@ -339,81 +414,6 @@ high_offset_selects → table affected: page
339414
Queries performed: 100
340415
```
341416

342-
## SQL query log
343-
344-
It's a text file with a single SQL query in each line (no line breaks are allowed). Lines that do start with `--` (SQL comment) are ignored. The file can be [generated using `query-digest` when `--sql-log` output mode is selected](https://github.com/macbre/query-digest#output-modes).
345-
346-
An example:
347-
348-
```sql
349-
-- A comment
350-
select * from 0002_not_used_indices order by id
351-
select * from 0002_not_used_indices where foo = 'foo' and id = 2
352-
select count(*) from 0002_not_used_indices where foo = 'foo'
353-
select * from 0002_not_used_indices where bar = 'foo'
354-
INSERT IGNORE INTO `0070_insert_ignore` VALUES ('123', 9, '2017-01-01');
355-
```
356-
357-
## Formatters
358-
359-
`index-digest` can return results in various formats (use `--format` to choose one).
360-
361-
### plain
362-
363-
Emits human-readable report to a console. You can disable colored and bold text by setting env variable `ANSI_COLORS_DISABLED=1`.
364-
365-
### syslog
366-
367-
Pushes JSON-formatted messages via syslog, so they can be aggregated using ELK stack.
368-
Use `SYSLOG_IDENT` env variable to customize syslog's `ident` messages are sent with (defaults to `index-digest`).
369-
370-
```
371-
Dec 28 15:59:58 debian index-digest[17485]: {"meta": {"version": "index-digest v0.1.0", "database_name": "index_digest", "database_host": "debian", "database_version": "MySQL v5.7.20"}, "report": {"type": "redundant_indices", "table": "0004_id_foo", "message": "\"idx\" index can be removed as redundant (covered by \"PRIMARY\")", "context": {"redundant": "UNIQUE KEY idx (id, foo)", "covered_by": "PRIMARY KEY (id, foo)", "schema": "CREATE TABLE `0004_id_foo` (\n `id` int(9) NOT NULL AUTO_INCREMENT,\n `foo` varbinary(16) NOT NULL DEFAULT '',\n PRIMARY KEY (`id`,`foo`),\n UNIQUE KEY `idx` (`id`,`foo`)\n) ENGINE=InnoDB DEFAULT CHARSET=latin1", "table_data_size_mb": 0.015625, "table_index_size_mb": 0.015625}}}
372-
```
373-
374-
### yaml
375-
376-
Outputs YML file with results and metadata.
377-
378-
## Checks
379-
380-
You can select which checks should be reported by the tool by using `--checks` command line option. Certain checks can also be skipped via `--skip-checks` option. Refer to `index_digest --help` for examples.
381-
382-
> **Number of checks**: 22
383-
384-
* `redundant_indices`: reports indices that are redundant and covered by other
385-
* `non_utf_columns`: reports text columns that have characters encoding set to `latin1` (utf is the way to go)
386-
* `missing_primary_index`: reports tables with no primary or unique key (see [MySQL bug #76252](https://bugs.mysql.com/bug.php?id=76252) and [Wikia/app#9863](https://github.com/Wikia/app/pull/9863))
387-
* `test_tables`: reports tables that seem to be test leftovers (e.g. `some_guy_test_table`)
388-
* `single_column`: reports tables with just a single column
389-
* `empty_tables`: reports tables with no rows
390-
* `generic_primary_key`: reports tables with [a primary key on `id` column](https://github.com/jarulraj/sqlcheck/blob/master/docs/logical/1004.md) (a more meaningful name should be used)
391-
* `use_innodb`: reports table using storage engines different than `InnoDB` (a default for MySQL 5.5+ and MariaDB 10.2+)
392-
393-
### Additional checks performed on SQL log
394-
395-
> You need to provide SQL log file via `--sql-log` option
396-
397-
* `not_used_columns`: checks which columns were not used by SELECT queries
398-
* `not_used_indices`: checks which indices are not used by SELECT queries
399-
* `not_used_tables`: checks which tables are not used by SELECT queries
400-
* `queries_not_using_index`: reports SELECT queries that do not use any index
401-
* `queries_using_filesort`: reports SELECT queries that require filesort ([a sort can’t be performed from an index and quicksort is used](https://www.percona.com/blog/2009/03/05/what-does-using-filesort-mean-in-mysql/))
402-
* `queries_using_temporary`: reports SELECT queries that require a temporary table to hold the result
403-
* `queries_using_full_table_scan`: reports SELECT queries that require a [full table scan](https://dev.mysql.com/doc/refman/5.7/en/table-scan-avoidance.html)
404-
* `selects_with_like`: reports SELECT queries that use `LIKE '%foo'` conditions (they can not use an index)
405-
* `insert_ignore`: reports [queries using `INSERT IGNORE`](https://medium.com/legacy-systems-diary/things-to-avoid-episode-1-insert-ignore-535b4c24406b)
406-
* `select_star`: reports [queries using `SELECT *`](https://github.com/jarulraj/sqlcheck/blob/master/docs/query/3001.md)
407-
* `having_clause`: reports [queries using `HAVING` clause](https://github.com/jarulraj/sqlcheck/blob/master/docs/query/3012.md)
408-
* `high_offset_selects`: report [SELECT queries using high OFFSET](https://www.percona.com/blog/2008/09/24/four-ways-to-optimize-paginated-displays/)
409-
410-
### Additional checks performed on tables data
411-
412-
> You need to use `--analyze-data` command line switch. Please note that these checks will query your tables. **These checks can take a while if queried columns are not indexed**.
413-
414-
* `data_too_old`: reports tables that have really old data, maybe it's worth checking if such long data retention is actually needed (**defaults to three months threshold**, can be customized via `INDEX_DIGEST_DATA_TOO_OLD_THRESHOLD_DAYS` env variable)
415-
* `data_not_updated_recently`: reports tables that were not updated recently, check if it should be up-to-date (**defaults a month threshold**, can be customized via `INDEX_DIGEST_DATA_NOT_UPDATED_RECENTLY_THRESHOLD_DAYS` env variable)
416-
417417
## Success stories
418418

419419
> Want to add your entry here? Submit a pull request

0 commit comments

Comments
 (0)