sql like 优化

A:
最多使用ffs,其他没有什么好方法!
两边都有%,就不会走index了。。
如果表不是很大,可以考虑keep之。。

B:
这个问题首先是在TAOBAO DBA的BLOG上看到丹臣写的关于Like和INSTR的性能问题。不过他只是给出了结果。我对这个函数性能感到有趣,之前一直没有关注过,遂自己详细测试了下。

Oracle 9208:

SQL> select count(*) from item;

COUNT(*)

----------

2781806

SQL> select count(*) from item where item like '%A0';

COUNT(*)

----------

9036

Elapsed: 00:00:04.03

Execution Plan

----------------------------------------------------------

0 SELECT STATEMENT Optimizer=CHOOSE (Cost=1134 Card=1 Bytes=16

)

1 0 SORT (AGGREGATE)

2 1 INDEX (FAST FULL SCAN) OF 'PK_ITEM' (UNIQUE) (Cost=1134

Card=138698 Bytes=2219168)

Statistics

----------------------------------------------------------

0 recursive calls

0 db block gets

10369 consistent gets

0 physical reads

0 redo size

519 bytes sent via SQL*Net to client

656 bytes received via SQL*Net from client

2 SQL*Net roundtrips to/from client

0 sorts (memory)

0 sorts (disk)

1 rows processed

可以看到走了FFS, 10396 Consistent Gets, Elapsed tail=4.03

再看SUBSTR

SQL> select count(*) from item where substr(item,-2)='A0';

COUNT(*)

----------

9036

Elapsed: 00:00:00.84

Execution Plan

----------------------------------------------------------

0 SELECT STATEMENT Optimizer=CHOOSE (Cost=1134 Card=1 Bytes=16

)

1 0 SORT (AGGREGATE)

2 1 INDEX (FAST FULL SCAN) OF 'PK_ITEM' (UNIQUE) (Cost=1134

Card=27740 Bytes=443840)

Statistics

----------------------------------------------------------

0 recursive calls

0 db block gets

10369 consistent gets

0 physical reads

0 redo size

519 bytes sent via SQL*Net to client

656 bytes received via SQL*Net from client

2 SQL*Net roundtrips to/from client

0 sorts (memory)

0 sorts (disk)

1 rows processed

可以看到PLAN一样,Consistent Gets一样,Elapsed tail=0.84

SUBSTR的耗时只有LIKE的21%左右,鉴于PLAN和Consistent Gets一致,所以可以认为是SUBSTR的CPU Cost要小于LIKE。

NOT LIKE的情况和LIKE相似。

INSTR的情况也如SUBSTR

确实如丹臣所言,Oracle的函数有相当的优化。

但9i的Optimizer在计算Cost的时候是以IO为准,那么在以CPU为准的10G上呢,Cost会有什么差别?

下面在10203上测试

SQL> select count(*) from item where item like '%A0';

COUNT(*)

----------

9104

Elapsed: 00:00:03.03

Execution Plan

----------------------------------------------------------

Plan hash value: 642095792

--------------------------------------------------------------------------------

-

| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time

|

--------------------------------------------------------------------------------

-

| 0 | SELECT STATEMENT | | 1 | 15 | 2211 (3)| 00:00:27

|

| 1 | SORT AGGREGATE | | 1 | 15 | |

|

|* 2 | INDEX FAST FULL SCAN| PK_ITEM | 139K| 2037K| 2211 (3)| 00:00:27

|

--------------------------------------------------------------------------------

-

Predicate Information (identified by operation id):

---------------------------------------------------

2 - filter("ITEM" LIKE '%A0')

Statistics

----------------------------------------------------------

0 recursive calls

0 db block gets

10117 consistent gets

0 physical reads

0 redo size

516 bytes sent via SQL*Net to client

492 bytes received via SQL*Net from client

2 SQL*Net roundtrips to/from client

0 sorts (memory)

0 sorts (disk)

1 rows processed

SQL> select count(*) from item where substr(item,-2)='A0';

COUNT(*)

----------

9104

Elapsed: 00:00:01.24

Execution Plan

----------------------------------------------------------

Plan hash value: 642095792

--------------------------------------------------------------------------------

-

| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time

|

----------------------------------------------------------------------

-

| 0 | SELECT STATEMENT | | 1 | 15 | 2220 (3)| 00:00:27

|

| 1 | SORT AGGREGATE | | 1 | 15 | |

|

|* 2 | INDEX FAST FULL SCAN| PK_ITEM | 27819 | 407K| 2220 (3)| 00:00:27

|

--------------------------------------------------------------------------------

-

Predicate Information (identified by operation id):

---------------------------------------------------

2 - filter(SUBSTR("ITEM",-2)='A0')

Statistics

----------------------------------------------------------

0 recursive calls

0 db block gets

10117 consistent gets

0 physical reads

0 redo size

516 bytes sent via SQL*Net to client

492 bytes received via SQL*Net from client

2 SQL*Net roundtrips to/from client

0 sorts (memory)

0 sorts (disk)

1 rows processed

结果是SUBSTR-2220-elapsed 1.24,Like-2211-elapsed 3.03。

显然SUBSTR在CPU占用上要高那么一点, 但是其相对于Like仅 41% 的耗时,使得它完全可以取代Like在查询以XX结尾的SQL中的地位。

BTW,我在一台Idle Server上的测试表明,使用SUBSTR不仅在耗时上缩短,而且CPU使用率也较LIKE低(9.5%/12.5%)

类似的测试表明INSTR相对LIKE ‘%XX%’的优势。

(函数取代LIKE ‘XX%’就别想了,一个Index Range Scan相对FFS的优势太大了)


C:
SQL> select count(*) from t;

  COUNT(*)
----------
     51838
SQL> insert /*+append*/ into t select * from t;
已创建51838行。
SQL> commit;
提交完成。
SQL> insert /*+append*/ into t select * from t;
已创建103676行。
SQL> commit;
提交完成。
SQL> insert /*+append*/ into t select * from t;
已创建207352行。
SQL> commit;
提交完成。
SQL> insert /*+append*/ into t select * from t;
已创建414704行。
SQL> commit;
提交完成。
SQL> select count(*) from t;

  COUNT(*)
----------
    829408
SQL> set autot traceonly exp stat
SQL> select * from t where object_type like '%TYPE%';

已选择34064行。

执行计划
----------------------------------------------------------
Plan hash value: 1601196873
--------------------------------------------------------------------------
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |  2592 |   235K|   160   (2)| 00:00:02 |
|*  1 |  TABLE ACCESS FULL| T    |  2592 |   235K|   160   (2)| 00:00:02 |
--------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
   1 - filter("OBJECT_TYPE" LIKE '%TYPE%')

统计信息
----------------------------------------------------------
          0  recursive calls
          0  db block gets
      13623  consistent gets
      11364  physical reads
          0  redo size
    1466572  bytes sent via SQL*Net to client
      25370  bytes received via SQL*Net from client
       2272  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      34064  rows processed
SQL> SQL> select /*+index(t,t_idx)*/ * from t where object_type like '%TYPE%';

已选择34064行。

执行计划
----------------------------------------------------------
Plan hash value: 3778778741
-------------------------------------------------------------------------------------
| Id  | Operation                   | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
-------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |       |  2592 |   235K|   260   (2)| 00:00:04 |
|   1 |  TABLE ACCESS BY INDEX ROWID| T     |  2592 |   235K|   260   (2)| 00:00:04 |
|*  2 |   INDEX FULL SCAN           | T_IDX |  2592 |       |   146   (2)| 00:00:02 |
-------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
   2 - filter("OBJECT_TYPE" LIKE '%TYPE%')

统计信息
----------------------------------------------------------
          1  recursive calls
          0  db block gets
      14131  consistent gets
       1982  physical reads
     290188  redo size
    1477872  bytes sent via SQL*Net to client
      25370  bytes received via SQL*Net from client
       2272  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      34064  rows processed
SQL> SQL>
SQL>
SQL>
SQL> select * from t where instr(object_type,'TYPE')>0;

已选择34064行。

执行计划
----------------------------------------------------------
Plan hash value: 1601196873
--------------------------------------------------------------------------
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |  2592 |   235K|   161   (3)| 00:00:02 |
|*  1 |  TABLE ACCESS FULL| T    |  2592 |   235K|   161   (3)| 00:00:02 |
--------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
   1 - filter(INSTR("OBJECT_TYPE",'TYPE')>0)

统计信息
----------------------------------------------------------
          0  recursive calls
          0  db block gets
      13623  consistent gets
       9952  physical reads
          0  redo size
    1466572  bytes sent via SQL*Net to client
      25370  bytes received via SQL*Net from client
       2272  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      34064  rows processed
SQL> SQL> SELECT * from t where rowid in(select /*+index_ffs(t,t_idx)*/ rowid from t where object_type like '%TYPE%');
已选择34064行。

执行计划
----------------------------------------------------------
Plan hash value: 628352769
-------------------------------------------------------------------------------
| Id  | Operation             | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
-------------------------------------------------------------------------------
|   0 | SELECT STATEMENT      |       |  2592 |   288K|   197   (4)| 00:00:03 |
|*  1 |  HASH JOIN            |       |  2592 |   288K|   197   (4)| 00:00:03 |
|*  2 |   INDEX FAST FULL SCAN| T_IDX |  2592 | 54432 |    35   (6)| 00:00:01 |
|   3 |   TABLE ACCESS FULL   | T     | 51838 |  4707K|   161   (3)| 00:00:02 |
-------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
   1 - access(ROWID=ROWID)
   2 - filter("OBJECT_TYPE" LIKE '%TYPE%')

统计信息
----------------------------------------------------------
          1  recursive calls
          0  db block gets
      17559  consistent gets
       9991  physical reads
          0  redo size
    1466572  bytes sent via SQL*Net to client
      25370  bytes received via SQL*Net from client
       2272  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      34064  rows processed



小结:

   1,目前看instr比rowid及全表报描的性能更高及index_ffs更高

    2,index_ffs虽强制走了索引,但成本最高

D:
free介绍的是一种方法;RudolfLu曾经也介绍过一种方法,就是根据搜索字段+rowid建立一个物理表,对这个物理表进行全表,相较ffs会更快一些,不过需要定期刷新物理表,适用于变化较少,查询要求实时性要求不高的环境。

其实也可以用Oracle的全文检索技术,可能会有更好的结果,看我以下的一个例子:

[php]

1.这是模糊查询的结果
SQL> select count(*) from jivemessage where subject like '%abc%';

  COUNT(*)
----------
        13

Elapsed: 00:00:15.31

2.这是全文检索的结果
SQL> select count(*) from jivemessage where contains(subject,'abc')>0;

  COUNT(*)
----------
        25

Elapsed: 00:00:00.03

3.模糊查询的内容
SQL> select subject from jivemessage where subject like '%abc%';

SUBJECT
--------------------------------------------------------------------------------
abc
测试发贴""abc
abc
Re: abc
Re: abc
Re: abc
<testabcde>
初夜abc
abcd
abc
abc

SUBJECT
--------------------------------------------------------------------------------
哈哇abc
http;//baoxing.168abc.com

13 rows selected.

Elapsed: 00:00:03.29

4.也许这是我们更想要的结果
SQL> select subject from jivemessage where contains(subject,'abc')>0;

SUBJECT
--------------------------------------------------------------------------------
【游戏】 把你的名字的首字母用智能ABC打出,看能出来什么?
游戏——把你的名字首字母用智能ABC打出来
智能ABC暗藏杀机
ABC
ABC
ABC
ABC
ABC
ABC
哈哇abc
abc

SUBJECT
--------------------------------------------------------------------------------
abc
ABC
振奋爱的激情方案ABC
智能ABC的错吗?
ABC全选
瓜果美容ABC
经典英文歌曲ABC,不好你拿版砖砍我,好就回帖顶一下!
初夜abc
Re: abc
Re: abc
Re: abc

SUBJECT
--------------------------------------------------------------------------------
abc
测试发贴""abc
abc

25 rows selected.

Elapsed: 00:00:00.04

D:
select * from foo where rowid in (select /*+index_ffs(foo ind_name) */rowid from foo where name like '%ddd%' )

e:
SELECT
       org.*
  FROM test_ffs org, (SELECT /*+no_merge index_ffs(test_ffs  object_name ) */
                             ROWID AS r, object_name
                        FROM test_ffs
                       WHERE object_name LIKE '%A%' ) tmp
WHERE org.ROWID = tmp.r ;
INDEX_FFS(table  index_name)所查询的列必须全部被索引才可使用


f:
1。eygle 的 全文检索法:
对于用于条件的column length 占TABLE的record length的大部分的,速度应该是最快的。如果用于条件的column length 占TABLE的record length的比例并不是很大,那样优势就不明显,毕竟要多安装和管理一个组件。

2。RudolfLu的新表法
和上面对应的是,如果用于条件的column length 占TABLE的record length的比例并不是很大,比较好用,如果用于条件的column length 占TABLE的record length的大部分的话,那就没什么用了,另外,要多维护一个table,还要考虑两个表同步的问题,不建议使用。

3。我的index_ffs + rowid 法
主要优劣势和 RudolfLu的新表法 接近,但是index可以自动更新。而且,在某种环境里,可能已经有相关的index,可能并不需要增加新的index.
另外,没做过测试,不知道相同数据量的index_ffs  和 FTS 性能区别。


g:
1、尽量不要使用 like '%..%'

2、对于 like '..%..' (不以 % 开头),Oracle可以应用 colunm上的index

3、对于 like '%...' 的 (不以 % 结尾),可以利用 reverse + function index 的形式,变化成 like '..%' 代码



建测试表和Index。

注意:重点在于带reverse的function index。同时,一定要使用CBO才行......



SQL> select reverse('123') from dual;

REVERSE('123')

--------------------------------

321

1 row selected.



SQL> create table test_like as select object_id,object_name from dba_objects;

Table created.



SQL> create index test_like__name on test_like(object_name);

Index created.



SQL> create index test_like__name_reverse on test_like(reverse(object_name));

Index created.



SQL> analyze table test_like compute statistics for table for all indexes;

Table analyzed.



SQL> set autot trace



--常量开头的like , 会利用index ,没问题......

SQL> select * from test_like where object_name like AS%';

Execution Plan

----------------------------------------------------------

0 SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=655 Bytes=15720)

1 0 TABLE ACCESS (BY INDEX ROWID) OF 'TEST_LIKE' (Cost=2 Card=655Bytes=15720)

2 1 INDEX (RANGE SCAN) OF 'TEST_LIKE__NAME' (NON-UNIQUE) (Cost=2 Card=118)



-- 开头和结尾都是%,对不起,很难优化



SQL> select * from test_like where object_name like '%%';



Execution Plan

----------------------------------------------------------

0 SELECT STATEMENT Optimizer=CHOOSE (Cost=6 Card=655 Bytes=15720)

1 0 TABLE ACCESS (FULL) OF 'TEST_LIKE' (Cost=6 Card=655 ytes=15720)



-- 以常量结束,直接写的时候是不能应用index的

SQL> select * from test_like where object_name like '%S';

Execution Plan

----------------------------------------------------------

0 SELECT STATEMENT Optimizer=CHOOSE (Cost=6 Card=655 Bytes=15720)

1 0 TABLE ACCESS (FULL) OF 'TEST_LIKE' (Cost=6 Card=655 Bytes=15720)



--'以常量结束的,加个reverse 函数,又可以用上index了'

SQL> select * from test_like where reverse(object_name)like reverse('%AS');

Execution Plan

----------------------------------------------------------

0 SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=655 Bytes=15720)

1 0 TABLE ACCESS (BY INDEX ROWID) OF 'TEST_LIKE' (Cost=2 Card=655 Bytes=15720)

2 1 INDEX (RANGE SCAN) OF 'TEST_LIKE__NAME_REVERSE' (NON-UNIQUE) (Cost=2 Card=118)

猜你喜欢

转载自housheng33.iteye.com/blog/1462091