安徽建设工程信息网站,广西建设工程协会网站,手机ftp上传wordpress,开发一个平台项目hbase数据库出现很诡异的assignment #xff0c;region移动的src和dest都是同一台regionserver#xff0c;不过时间戳不同#xff0c;启动的只有一个regionserver#xff0c; 不知道怎么出现了两个时间戳 分析下源码解决一下 loadbalance只有一个实现 org.apache.hado…项目hbase数据库出现很诡异的assignment region移动的src和dest都是同一台regionserver不过时间戳不同启动的只有一个regionserver 不知道怎么出现了两个时间戳 分析下源码解决一下  loadbalance只有一个实现 org.apache.hadoop.hbase.master.DefaultLoadBalancer 在HMaster中会启动一个线程 org.apache.hadoop.hbase.Chore按照设定的hbase.balancer.period默认300000ms五分钟遍历所有表根据每个表在regionserver中的region数量做balance有一个平衡系数hbase.regions.slop默认0.2根据region总数算出平均region值avg×0.8 取整作为最小值avg×1.2取整作为最大值regionserver上超过最大值要移走小于最小值要移动region过来。否则打印目前的平衡状态。 assignmentManager 根据上述步骤生成的RegionPlan, 从src移动region到desc  src和desc都是ServerName对象 HMaster启动时会等待region servers注册到serverManager // Wait for region servers to report in. this.serverManager.waitForRegionServers(status); // Check zk for regionservers that are up but didnt register for (ServerName sn: this.regionServerTracker.getOnlineServers()) {if (!this.serverManager.isServerOnline(sn)) {// Not registered; add it.LOG.info(Registering server found up in zk but who has not yet reported in: sn);this.serverManager.recordNewServer(sn, HServerLoad.EMPTY_HSERVERLOAD);} } serverManager线程sleep一定时间等待HRegionServer注册 HRegionServer.java: // Try and register with the Master; tell it we are here. Break if// server is stopped or the clusterup flag is down or hdfs went wacky.while (keepLooping()) {MapWritable w reportForDuty();if (w null) {LOG.warn(reportForDuty failed; sleeping and then retrying.);this.sleeper.sleep();} else {handleReportForDutyResponse(w);break;}} HRegionServer 注册之后进入mainloop // The main run loop.while (!this.stopped isHealthy()) {　　long now System.currentTimeMillis(); 　　　if ((now - lastMsg) msgInterval) { 　　　　　　doMetrics(); 　　　　　　tryRegionServerReport(); 　　　　　　lastMsg System.currentTimeMillis(); 　　　　}   } 每隔hbase.regionserver.msginterval时间默认3秒进行一次注册尝试,如果服务器ip和端口不在已注册列表中则添加ServerName进map ServerManager.java void regionServerReport(ServerName sn, HServerLoad hsl)throws YouAreDeadException, PleaseHoldException {checkIsDead(sn, REPORT);if (!this.onlineServers.containsKey(sn)) {// Already have this hostport combo and its just different start code?checkAlreadySameHostPort(sn);// Just let the server in. Presume master joining a running cluster.// recordNewServer is what happens at the end of reportServerStartup.// The only thing we are skipping is passing back to the regionserver// the ServerName to use. Here we presume a master has already done// that so well press on with whatever it gave us for ServerName.recordNewServer(sn, hsl);} else {this.onlineServers.put(sn, hsl);}} recordNewServer 会打印 ServerName对象的ip 端口和时间戳信息 同一个region server注册的ServerName对象 会拥有同样的时间戳  this.startcode System.currentTimeMillis();...result this.hbaseMaster.regionServerStartup(port, this.startcode, now);...this.serverNameFromMasterPOV new ServerName(hostnameFromMasterPOV, this.isa.getPort(), this.startcode);...this.hbaseMaster.regionServerReport(this.serverNameFromMasterPOV.getVersionedBytes(), hsl);   region server启动时startCode是固定死的按照这个流程是不会出现相同IP和端口但时间戳不同的region server跑在线上的  如果一台机器上启动了两个region server 会把时间戳小的移出下次添加进时间戳大的进去 我们遇到的问题是时间戳不同的regionserver被注册在了master上并且相互之间做region move   转载于:https://www.cnblogs.com/shenguanpu/archive/2012/07/30/2615214.html