I found a session waiting for an event of Library Cache Lock. In the first place, I thought there could be blocking sessions that locked the objects. But I found no session holding the handler of the lock.
There's a report talking about this phenomenon.
Bug 2666244 - Library cache lock corruption possible (Doc ID 2666244.8)
This problem is introduced in 9.2.0.1. Library cache lock corruption is possible. This can result in processes waiting for a library cache lock without a holder, incorrect lock mode held by processes.
Luckily, I found there were several abnormal processes taking considerable CPU resource, which sessions should be killed a few days ago. So I killed these zombie processes on OS level, then the lock released. That is to say, we may not identify the holder of the lock by tracing it in v$lock, v$session or trace file, but we can try to identify suspects to kill instead. Or possibly, restarting the database service as a last resort.