转自 :
最近学习了Http连接池 - 五月的仓颉 - 博客园
【1】使用线程池与否的程序性能
我的任务定义:从0 累加到 100w;
public class ThreadPoolMain {/*** 线程池测试*/private static final AtomicInteger THREAD_EXECUTED_TOTAL = new AtomicInteger(0); // 已执行线程总数private static final AtomicLong EXECUTE_COST_MS = new AtomicLong(0); // 执行耗时毫秒private static final Integer ACCUMULATED_SUM_UPPER = 1000000; // 单个任务累加和上限private static final Integer TASK_TOTAL = 1000; // 任务总计private class IncreaseThread implements Runnable {public void run() {long startTime = System.currentTimeMillis();AtomicInteger counter = new AtomicInteger(0);for (int i = 0; i < ACCUMULATED_SUM_UPPER; i++) {counter.incrementAndGet();}// 累加执行时间EXECUTE_COST_MS.addAndGet(System.currentTimeMillis() - startTime);if (THREAD_EXECUTED_TOTAL.incrementAndGet() == TASK_TOTAL) {System.out.println("cost: " + EXECUTE_COST_MS.get() + "ms");}}}
【1.1】方式1,不使用线程池
来一个任务,就开启线程运行它;
/*** 不使用线程池* 来一个任务,就开启线程运行它*/@Testpublic void testRunWithoutThreadPool() {List<Thread> tList = new ArrayList<Thread>(TASK_TOTAL);for (int i = 0; i < TASK_TOTAL; i++) {tList.add(new Thread(new IncreaseThread()));}for (Thread t : tList) {t.start();}for (;;);}
// cost: 2418156 ms
【1.2】方式2,使用线程池
创建包含多个线程的线程池,来一个任务,从线程池中取出线程运行任务,而不是重新创建一个;
/*** 使用线程池*/@Testpublic void testRunWithThreadPool() {ThreadPoolExecutor executor = new ThreadPoolExecutor(100, 100, 0, TimeUnit.MILLISECONDS,new LinkedBlockingQueue());for (int i = 0; i < TASK_TOTAL; i++) {executor.submit(new IncreaseThread());}for (;;);}
// cost: 27306 ms
小结: 很显然,使用线程池后,运行任务的速度更快,性能更高;
主要原因在于 方式1不使用线程池,创建线程成本累加起来非常高,耗费资源多,然后这些资源却没有真正执行业务逻辑计算;
【2】http连接池
1)http 运行架构
/*** 连接池基类*/
public class BaseHttpClientTest {protected static final int REQUEST_COUNT = 5;protected static final String SEPERATOR = " ";protected static final AtomicInteger NOW_COUNT = new AtomicInteger(0);protected static final StringBuilder EVERY_REQ_COST = new StringBuilder(200);/*** 获取待运行的线程*/protected List<Thread> getRunThreads(Runnable runnable) {List<Thread> tList = new ArrayList<Thread>(REQUEST_COUNT);for (int i = 0; i < REQUEST_COUNT; i++) {tList.add(new Thread(runnable));}return tList;}/*** 启动所有线程*/protected void startUpAllThreads(List<Thread> tList) {for (Thread t : tList) {t.start();// 这里需要加一点延迟,保证请求按顺序发出去try {Thread.sleep(300);} catch (InterruptedException e) {e.printStackTrace();}}}/*** 计算执行成本,包括耗时*/protected synchronized void addCost(long cost) {EVERY_REQ_COST.append(cost);EVERY_REQ_COST.append("ms");EVERY_REQ_COST.append(SEPERATOR);}
}
【2.1】方式1,不使用连接池发送http请求
/*** 不使用连接池测试*/
public class HttpWithoutPoolTest extends BaseHttpClientTest {@Testpublic void test() throws Exception {startUpAllThreads(getRunThreads(new HttpThread()));// 等待线程运行for (;;);}private class HttpThread implements Runnable {public void run() {/*** HttpClient是线程安全的,因此HttpClient正常使用应当做成全局变量,但是一旦全局共用一个,HttpClient内部构建的时候会new一个连接池* 出来,这样就体现不出使用连接池的效果,因此这里每次new一个HttpClient,保证每次都不通过连接池请求对端*/CloseableHttpClient httpClient = HttpClients.custom().build();HttpGet httpGet = new HttpGet("https://www.baidu.com/");long startTime = System.currentTimeMillis();try {CloseableHttpResponse response = httpClient.execute(httpGet);if (response != null) {response.close();}} catch (Exception e) {e.printStackTrace();} finally {addCost(System.currentTimeMillis() - startTime);if (NOW_COUNT.incrementAndGet() == REQUEST_COUNT) {System.out.println(EVERY_REQ_COST.toString());}}}}
}
// 460ms 461ms 462ms 462ms 462ms
【2.2】方式2,使用连接池发送http请求
/*** 使用连接池测试*/
public class HttpWithPoolTest extends BaseHttpClientTest {private CloseableHttpClient httpClient = null;@Beforepublic void before() {initHttpClient();}@Testpublic void test() throws Exception {startUpAllThreads(getRunThreads(new HttpThread()));// 等待线程运行for (;;);}private class HttpThread implements Runnable {public void run() {HttpGet httpGet = new HttpGet("https://www.baidu.com/");// 长连接标识,不加也没事,HTTP1.1默认都是Connection: keep-alive的httpGet.addHeader("Connection", "keep-alive");long startTime = System.currentTimeMillis();try {CloseableHttpResponse response = httpClient.execute(httpGet);if (response != null) {response.close();}} catch (Exception e) {e.printStackTrace();} finally {addCost(System.currentTimeMillis() - startTime);if (NOW_COUNT.incrementAndGet() == REQUEST_COUNT) {System.out.println(EVERY_REQ_COST.toString());}}}}private void initHttpClient() {final PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();// 总连接池数量connectionManager.setMaxTotal(1);// 可为每个域名设置单独的连接池数量connectionManager.setMaxPerRoute(new HttpRoute(new HttpHost("www.baidu.com")), 1);// setConnectTimeout表示设置建立连接的超时时间// setConnectionRequestTimeout表示从连接池中拿连接的等待超时时间// setSocketTimeout表示发出请求后等待对端应答的超时时间RequestConfig requestConfig = RequestConfig.custom().setConnectTimeout(1000).setConnectionRequestTimeout(2000).setSocketTimeout(3000).build();// 重试处理器,StandardHttpRequestRetryHandler这个是官方提供的,看了下感觉比较挫,很多错误不能重试,可自己实现HttpRequestRetryHandler接口去做HttpRequestRetryHandler retryHandler = new StandardHttpRequestRetryHandler();httpClient = HttpClients.custom().setConnectionManager(connectionManager).setDefaultRequestConfig(requestConfig).setRetryHandler(retryHandler).build();// 服务端假设关闭了连接,对客户端是不透明的,HttpClient为了缓解这一问题,在某个连接使用前会检测这个连接是否过时,如果过时则连接失效,但是这种做法会为每个请求// 增加一定额外开销,因此有一个定时任务专门回收长时间不活动而被判定为失效的连接,可以某种程度上解决这个问题Timer timer = new Timer();timer.schedule(new TimerTask() {@Overridepublic void run() {try {// 关闭失效连接并从连接池中移除connectionManager.closeExpiredConnections();// 关闭30秒钟内不活动的连接并从连接池中移除,空闲时间从交还给连接管理器时开始connectionManager.closeIdleConnections(20, TimeUnit.SECONDS);} catch (Throwable t) {t.printStackTrace();}}}, 0 , 1000 * 5);}
}
// 346ms 205ms 184ms 167ms 169ms
小结: 使用 http连接池发送请求的性能, 优于不使用http连接 ;
更深层次原因,refer2
最近学习了Http连接池 - 五月的仓颉 - 博客园
这张图特别有意思,可以看到发送请求与接收响应的详情