使用目的

我想要解决的问题是以最快的速度抽取PDF中的图片，再和对应文本进行关联，最终适配到LangChain上
经过调研pdf2image的covert_from_byte的sthread_count 参数，可以启动多线程会大大加快转换速度

我的系统是openEuler，命令使用和CentOS一样

下载源码

https://poppler.freedesktop.org/
我选择的是 poppler-24.01.0.tar.xz

安装依赖

直接安装的依赖

sudo yum install nss-devel fontconfig-devel libfreetype6-dev libtiff-devel mesa-libGL-devel ninja-build systemd-devel pcre2 pcre2-devel glib2 glib2-devel autoconf automake libtool harfbuzz-devel lcms2-devel libcurl-devel poppler-cpp-devel

需要编译的依赖

1、libassuan

wget https://www.gnupg.org/ftp/gcrypt/libassuan/libassuan-2.5.5.tar.bz2
tar xjf libassuan-2.5.5.tar.bz2
cd libassuan-2.5.5
./configure --prefix=/usr/local
make -j4
sudo make install

2、gpgme

wget https://www.gnupg.org/ftp/gcrypt/gpgme/gpgme-1.19.0.tar.bz2
tar xjf gpgme-1.19.0.tar.bz2
cd gpgme-1.19.0
./configure --prefix=/usr/local
make -j4
sudo make install

3、libb2

git clone https://github.com/BLAKE2/libb2.git
cd libb2
autoreconf -fi
./configure
make
sudo find / -name libb2.pc 2>/dev/null
# /home/HwHiAiUser/work/pdf_to_sql/BLAKE2/libb2/libb2.pc
export PKG_CONFIG_PATH=/path/to/libb2:$PKG_CONFIG_PATH

4、pcre2

rm /home/anaconda3/cmake/pcre2-config.cmake
rm /home/anaconda3/cmake/pcre2-config-version.cmake # 删除虚拟环境中的，防止后面干扰
rpm -ql pcre2-devel # 查看pcre2的安装位置，方便后面设置路径，参考：
# -DPCRE2_INCLUDE_DIRS=/usr/include -DPCRE2_LIBRARY=/usr/lib64/libpcre2.so

5、Qt6

下载qt-everywhere-src-6.2.4.tar.xz

tar xf qt-everywhere-src-6.2.4.tar.xz
cd qt-everywhere-src-6.2.4
vim CMakeLists.txt
# 在第4行开始加入
set(CMAKE_BUILD_WITH_INSTALL_RPATH TRUE)
set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib;/usr/lib64;/usr/lib;/home/anaconda3/lib")
# 保存后运行
./configure -DPCRE2_INCLUDE_DIRS=/usr/include -DPCRE2_LIBRARY=/usr/lib64/libpcre2.so -DPCRE2_FIND_COMPONENTS="8BIT;16BIT"
# 运行成功后显示
# -- Build files have been written to: /home/HwHiAiUser/work/pdf_to_sql/qt-everywhere-src-6.2.4
cmake --build . --parallel
make

命令行编译命令-poppler

我无法编译成功QT，五花八门的报错
所以只能关闭QT特性

cd poppler-24.01.0.tar.xz
vim CMakeCache.txt
ENABLE_QT5:BOOL=OFF 
ENABLE_QT6:BOOL=OFF

一共4个全部从NO改为OFF

mkdir build
cd build 
cmake .. 
make 
make install

最终成功安装

export LD_LIBRARY_PATH=/usr/local/lib64:$LD_LIBRARY_PATH

添加一下路径

pdfinfo

查看版本输出

pdfinfo version 24.01.0
Copyright 2005-2024 The Poppler Developers - http://poppler.freedesktop.org
Copyright 1996-2011, 2022 Glyph & Cog, LLC
Usage: pdfinfo [options] <PDF-file>-f <int>             : first page to convert-l <int>             : last page to convert-box                 : print the page bounding boxes-meta                : print the document metadata (XML)-custom              : print both custom and standard metadata-js                  : print all JavaScript in the PDF-struct              : print the logical document structure (for tagged files)-struct-text         : print text contents along with document structure (for tagged files)-isodates            : print the dates in ISO-8601 format-rawdates            : print the undecoded date strings directly from the PDF file-dests               : print all named destinations in the PDF-url                 : print all URLs inside PDF objects (does not scan text content)-enc <string>        : output text encoding name-listenc             : list available encodings-opw <string>        : owner password (for encrypted files)-upw <string>        : user password (for encrypted files)-v                   : print copyright and version info-h                   : print usage information-help                : print usage information--help               : print usage information-?                   : print usage information

测试

import pdfplumber
import PyPDF2
import pdf2image
from pdf2image import convert_from_path,convert_from_bytes
import tempfile
from pdf2image.exceptions import PDFInfoNotInstalledError, PDFPageCountError, PDFSyntaxError
import osfile_path = r'xxx.pdf' # PDF 文件路径
dir_path = r'output' # 存放图片的文件夹def pdf2image2(file_path, dir_path):images = convert_from_path(file_path, dpi=300)for image in images:if not os.path.exists(dir_path):os.makedirs(dir_path)image.save(dir_path + f'\img_{images.index(image)}.png', 'PNG')pdf2image2(file_path, dir_path)

小结

也许直接安装poppler-cpp-devel就可以成功，但是我是最后才安装的
pdf2image就是需要libpoppler.so.133，我现在也不清楚/usr/local/lib64中的libpoppler.so.133是poppler-cpp-devel安装的，还是编译poppler安装的了

sudo find / -name libpoppler.so.133
/usr/local/lib64/libpoppler.so.133
/home/HwHiAiUser/work/pdf_to_sql/poppler-24.01.0/build/libpoppler.so.133
export LD_LIBRARY_PATH=/usr/local/lib64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/home/HwHiAiUser/work/pdf_to_sql/poppler-24.01.0/build:$LD_LIBRARY_PATH

如果上述命令无效

vim /etc/ld.so.conf
# 添加：/usr/local/lib64
sudo ldconfig

编译QT6时出现了报错error: macro names must be identifiers
意义：宏名称必须是标识符，C++和C中命名变量时不能以数字开头，定义宏时也不能
在qt-everywhere-src-6.2.4/qtbase/src/3rdparty这个文件这里，就是数字开头，我尝试修改，无效
还有的报错是说磁盘空间不够，xxx包不支持等，依赖太多了