支付网站怎么设计的,下载168网站,福田瑞沃自卸车官网,wordpress网站源码分享简介 “今天发现一个比较奇怪的现象#xff0c;因为博主不熟悉mysql#xff0c;所以在安装hive的使用了postgresql作为hive的元数据库#xff0c;在测试几个连接工具对hive进行链接#xff0c;后面再测试的时候发现链接不上了#xff0c;并且报错日志如下#xff1a;” …简介 “今天发现一个比较奇怪的现象因为博主不熟悉mysql所以在安装hive的使用了postgresql作为hive的元数据库在测试几个连接工具对hive进行链接后面再测试的时候发现链接不上了并且报错日志如下” SLF4J: Failed toString() invocation on an object of type [org.apache.hadoop.hive.ql.session.OperationLog] java.lang.OutOfMemoryError: GC overhead limit exceeded FAILED: command has been interrupted: during query execution: null 2024-03-16 15:20:55,302 Log4j2-TF-2-AsyncLogger[AsyncContext71be98f5]-1 ERROR RandomAccessFileManager (/tmp/hadoop/operation_logs/513aee2e-444c-407e-8e5d-2f8541a9d2f7/hadoop_20240316152052_4af3ebb2-0011-48e8-acc3-46f60e08f82f) java.io.FileNotFoundException: /tmp/hadoop/operation_logs/513aee2e-444c-407e-8e5d-2f8541a9d2f7/hadoop_20240316152052_4af3ebb2-0011-48e8-acc3-46f60e08f82f (No such file or directory) java.io.FileNotFoundException: /tmp/hadoop/operation_logs/513aee2e-444c-407e-8e5d-2f8541a9d2f7/hadoop_20240316152052_4af3ebb2-0011-48e8-acc3-46f60e08f82f (No such file or directory)at java.io.RandomAccessFile.open0(Native Method)at java.io.RandomAccessFile.open(RandomAccessFile.java:316)at java.io.RandomAccessFile.init(RandomAccessFile.java:243)at java.io.RandomAccessFile.init(RandomAccessFile.java:124)at org.apache.logging.log4j.core.appender.RandomAccessFileManager$RandomAccessFileManagerFactory.createManager(RandomAccessFileManager.java:208)at org.apache.logging.log4j.core.appender.RandomAccessFileManager$RandomAccessFileManagerFactory.createManager(RandomAccessFileManager.java:186)at org.apache.logging.log4j.core.appender.AbstractManager.getManager(AbstractManager.java:114)at org.apache.logging.log4j.core.appender.OutputStreamManager.getManager(OutputStreamManager.java:100)at org.apache.logging.log4j.core.appender.RandomAccessFileManager.getFileManager(RandomAccessFileManager.java:72)at org.apache.hadoop.hive.ql.log.HushableRandomAccessFileAppender.createAppender(HushableRandomAccessFileAppender.java:178)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:136)at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:1120)at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:1045)at org.apache.logging.log4j.core.appender.routing.RoutingAppender.createAppender(RoutingAppender.java:299)at org.apache.logging.log4j.core.appender.routing.RoutingAppender.getControl(RoutingAppender.java:271)at org.apache.logging.log4j.core.appender.routing.RoutingAppender.append(RoutingAppender.java:229)at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:161)at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:134)at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:125)at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:89)at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:542)at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:500)at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:483)at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:471)at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:98)at org.apache.logging.log4j.core.async.AsyncLogger.actualAsyncLog(AsyncLogger.java:488)at org.apache.logging.log4j.core.async.RingBufferLogEvent.execute(RingBufferLogEvent.java:156)at org.apache.logging.log4j.core.async.RingBufferLogEventHandler.onEvent(RingBufferLogEventHandler.java:51)at org.apache.logging.log4j.core.async.RingBufferLogEventHandler.onEvent(RingBufferLogEventHandler.java:29)at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:129)at java.lang.Thread.run(Thread.java:750)2024-03-16 15:20:55,433 Log4j2-TF-2-AsyncLogger[AsyncContext71be98f5]-1 ERROR Unable to invoke factory method in class org.apache.hadoop.hive.ql.log.HushableRandomAccessFileAppender for element HushableMutableRandomAccess: java.lang.IllegalStateException: ManagerFactory [org.apache.logging.log4j.core.appender.RandomAccessFileManager$RandomAccessFileManagerFactory307957f1] unable to create manager for [/tmp/hadoop/operation_logs/513aee2e-444c-407e-8e5d-2f8541a9d2f7/hadoop_20240316152052_4af3ebb2-0011-48e8-acc3-46f60e08f82f] with data [org.apache.logging.log4j.core.appender.RandomAccessFileManager$FactoryData57fffa4] java.lang.reflect.InvocationTargetExceptionat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:136)at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:1120)at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:1045)at org.apache.logging.log4j.core.appender.routing.RoutingAppender.createAppender(RoutingAppender.java:299)at org.apache.logging.log4j.core.appender.routing.RoutingAppender.getControl(RoutingAppender.java:271)at org.apache.logging.log4j.core.appender.routing.RoutingAppender.append(RoutingAppender.java:229)at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:161)at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:134)at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:125)at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:89)at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:542)at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:500)at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:483)at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:471)at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:98)at org.apache.logging.log4j.core.async.AsyncLogger.actualAsyncLog(AsyncLogger.java:488)at org.apache.logging.log4j.core.async.RingBufferLogEvent.execute(RingBufferLogEvent.java:156)at org.apache.logging.log4j.core.async.RingBufferLogEventHandler.onEvent(RingBufferLogEventHandler.java:51)at org.apache.logging.log4j.core.async.RingBufferLogEventHandler.onEvent(RingBufferLogEventHandler.java:29)at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:129)at java.lang.Thread.run(Thread.java:750) Caused by: java.lang.IllegalStateException: ManagerFactory [org.apache.logging.log4j.core.appender.RandomAccessFileManager$RandomAccessFileManagerFactory307957f1] unable to create manager for [/tmp/hadoop/operation_logs/513aee2e-444c-407e-8e5d-2f8541a9d2f7/hadoop_20240316152052_4af3ebb2-0011-48e8-acc3-46f60e08f82f] with data [org.apache.logging.log4j.core.appender.RandomAccessFileManager$FactoryData57fffa4]at org.apache.logging.log4j.core.appender.AbstractManager.getManager(AbstractManager.java:116)at org.apache.logging.log4j.core.appender.OutputStreamManager.getManager(OutputStreamManager.java:100)at org.apache.logging.log4j.core.appender.RandomAccessFileManager.getFileManager(RandomAccessFileManager.java:72)at org.apache.hadoop.hive.ql.log.HushableRandomAccessFileAppender.createAppender(HushableRandomAccessFileAppender.java:178)... 25 more2024-03-16 15:20:55,695 Log4j2-TF-2-AsyncLogger[AsyncContext71be98f5]-1 ERROR Unable to create Appender of type HushableMutableRandomAccess Exception in thread HiveServer2-Handler-Pool: Thread-470 java.lang.OutOfMemoryError: GC overhead limit exceeded WARN: The method class org.apache.commons.logging.impl.SLF4JLogFactory#release() was invoked. WARN: Please see http://www.slf4j.org/codes.html#release for an explanation. 大概意思就是我JVM的资源不足了内存溢出了 这个就很奇怪 我就测试了几个数据库链接工具还没上强度也没上job怎么就内存溢出了呢 我查看了一下进程看看是什么事务占用了资源吓我一跳 每一次在测试的时候都会产生一个链接会话且一直没有被回收且即便是我关闭了会话这个会话进程依然存在长此以往会话占用的资源就爆满了。再有新的链接想要链接进来就会被refused。 这里和postgresql的底层设计有点关联postgresql是一个多进程多线程类型的一个数据库对于hive在安装的时候其实内置的是derby数据库。这里建议采用其默认数据库其兼容性会更好。目前行业主流仍然使用mysql作为其元数据库管理最多