在你的JDBC应用中使用IN列表填充以避免游标缓存争用问题

很少有开发者意识到一个问题,那就是在SQL中使用IN列表时可能会遇到 "游标缓存争夺 "或 "执行计划缓存争夺 "的问题。在以前的文章中详细描述了这个问题,可以概括为这样。

所有这些都是不同的SQL查询,需要在有强大计划缓存的RDBMS(例如Db2、Oracle、SQL Server)中被解析/计划/缓存为可能不同的执行计划:

SELECT * FROM t WHERE id IN (?);
SELECT * FROM t WHERE id IN (?, ?);
SELECT * FROM t WHERE id IN (?, ?, ?);
SELECT * FROM t WHERE id IN (?, ?, ?, ?);
SELECT * FROM t WHERE id IN (?, ?, ?, ?, ?);

虽然这在开发者机器上不是 问题,但在生产中会产生重大 问题。我见过这种情况,在高峰负载期间,整个Oracle实例都会瘫痪。虽然RDBMS供应商应该努力避免这可能导致的严重问题,但你可以使用我们在jOOQ发明的一个技巧来解决它(Hibernate现在也有)。

IN 列表填充

这个技巧非常简单。只要将你的IN 列表 "填充 "到最接近的2 的幂数,然后重复最后一个值,直到结束:

SELECT * FROM t WHERE id IN (?);                      -- Left as it is
SELECT * FROM t WHERE id IN (?, ?);                   -- Left as it is
SELECT * FROM t WHERE id IN (?, ?, ?, ?);             -- Padded 3 to 4
SELECT * FROM t WHERE id IN (?, ?, ?, ?);             -- Left as it is
SELECT * FROM t WHERE id IN (?, ?, ?, ?, ?, ?, ?, ?); -- Padded 5 to 8

这真的是一个黑客,有更好的解决方案来避免这个问题,包括使用数组或临时表,但你的生产系统可能已经瘫痪了,你需要一个快速修复。

从jOOQ 3.9(2016年底)开始,jOOQ已经支持IN 列表填充多年了,但随着相对较新的解析器和ParsingConnection ,你现在也可以在你的非jOOQ系统中对任何任意的SQL查询应用这一技术。这里有一个简单的例子:

// Any arbitrary JDBC Connection is wrapped by jOOQ here and replaced
// by a "ParsingConnection", which is also a JDBC Connection
DSLContext ctx = DSL.using(connection);
ctx.settings().setInListPadding(true);
Connection c = ctx.parsingConnection();

// Your remaining code is left untouched. It is unaware of jOOQ
for (int i = 0; i < 10; i++) {
    try (PreparedStatement s = c.prepareStatement(

        // This alone is reason enough to use jOOQ instead, 
        // but one step at a time :)
        "select 1 from dual where 1 in (" +
            IntStream.rangeClosed(0, i)
                     .mapToObj(x -> "?")
                     .collect(Collectors.joining(", ")) +
        ")")
    ) {
        for (int j = 0; j <= i; j++)
            s.setInt(j + 1, j + 1);

        try (ResultSet rs = s.executeQuery()) {
            while (rs.next())
                System.out.println(rs.getInt(1));
        }
    }
}

上面的例子只是生成和运行了10个这种形式的查询:

select 1 from dual where 1 in (?)
select 1 from dual where 1 in (?, ?)
select 1 from dual where 1 in (?, ?, ?)
select 1 from dual where 1 in (?, ?, ?, ?)
select 1 from dual where 1 in (?, ?, ?, ?, ?)
select 1 from dual where 1 in (?, ?, ?, ?, ?, ?)
select 1 from dual where 1 in (?, ?, ?, ?, ?, ?, ?)
select 1 from dual where 1 in (?, ?, ?, ?, ?, ?, ?, ?)
select 1 from dual where 1 in (?, ?, ?, ?, ?, ?, ?, ?, ?)
select 1 from dual where 1 in (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)

但这并不是正在执行的内容。在DEBUG日志中,我们可以看到以下内容:

Translating from         : select 1 from dual where 1 in (?)
Translating to           : select 1 from dual where 1 in (?)
Translating from         : select 1 from dual where 1 in (?, ?)
Translating to           : select 1 from dual where 1 in (?, ?)
Translating from         : select 1 from dual where 1 in (?, ?, ?)
Translating to           : select 1 from dual where 1 in (?, ?, ?, ?)
Translating from         : select 1 from dual where 1 in (?, ?, ?, ?)
Translating to           : select 1 from dual where 1 in (?, ?, ?, ?)
Translating from         : select 1 from dual where 1 in (?, ?, ?, ?, ?)
Translating to           : select 1 from dual where 1 in (?, ?, ?, ?, ?, ?, ?, ?)
Translating from         : select 1 from dual where 1 in (?, ?, ?, ?, ?, ?)
Translating to           : select 1 from dual where 1 in (?, ?, ?, ?, ?, ?, ?, ?)
Translating from         : select 1 from dual where 1 in (?, ?, ?, ?, ?, ?, ?)
Translating to           : select 1 from dual where 1 in (?, ?, ?, ?, ?, ?, ?, ?)
Translating from         : select 1 from dual where 1 in (?, ?, ?, ?, ?, ?, ?, ?)
Translating to           : select 1 from dual where 1 in (?, ?, ?, ?, ?, ?, ?, ?)
Translating from         : select 1 from dual where 1 in (?, ?, ?, ?, ?, ?, ?, ?, ?)
Translating to           : select 1 from dual where 1 in (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
Translating from         : select 1 from dual where 1 in (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
Translating to           : select 1 from dual where 1 in (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)

就这样,我们的传统应用程序又可以在生产中运行了,你将有时间来更彻底地解决这个问题。

结论

虽然jOOQ主要是一个内部DSL,用于在Java中编写类型安全的嵌入式SQL,但你也可以在任何基于JDBC的应用程序中使用它做很多其他事情。上面的例子是使用ParsingConnection ,它可以解析你所有的SQL语句,并将它们翻译/转换为任何其他东西,包括其他方言。

猜你喜欢

转载自juejin.im/post/7126371976599978020