主要是完成本地的Kafka和Druid搭建过程中，出现的Cannot start up because port 2181 is already in use.问题

问题原因：使用brew安装的的Druid也是自带zookeeper的，所以在单机部署的时候会和brew安装的Kafka的zookeeper端口2181冲突，如果两个一起启动，那就就需要修改Druid或者Kafka中的zookeeper端口为2182了

方法1：从执行逻辑上修改
/usr/local/Cellar/apache-druid-0.20.0/bin> ./start-micro-quickstart
Cannot start up because port 2181 is already in use.

If you need to change your ports away from the defaults, check out the
configuration documentation:

  https://druid.apache.org/docs/latest/configuration/index.html

If you believe this check is in error, or if you have changed your ports away
from the defaults, you can skip this check using an environment variable:

  export DRUID_SKIP_PORT_CHECK=1

直接启动会报错，这里提示端口被占用了,直接扫执行文件

/usr/local/Cellar/apache-druid-0.20.0/bin> cat start-micro-quickstart
#!/bin/bash -eu

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.

PWD="$(pwd)"
WHEREAMI="$(dirname "$0")"
WHEREAMI="$(cd "$WHEREAMI" && pwd)"

exec "$WHEREAMI/supervise" -c "$WHEREAMI/../conf/supervise/single-server/micro-quickstart.conf"

看到显示加载的conf文件是micro-quickstart.conf，直接查看该conf文件

/usr/local/Cellar/apache-druid-0.20.0/conf/supervise/single-server> cat micro-quickstart.conf
:verify bin/verify-java
:verify bin/verify-default-ports
:kill-timeout 10

!p10 zk bin/run-zk conf
coordinator-overlord bin/run-druid coordinator-overlord conf/druid/single-server/micro-quickstart
broker bin/run-druid broker conf/druid/single-server/micro-quickstart
router bin/run-druid router conf/druid/single-server/micro-quickstart
historical bin/run-druid historical conf/druid/single-server/micro-quickstart
!p90 middleManager bin/run-druid middleManager conf/druid/single-server/micro-quickstart

发现先去端口验证verify bin/verify-default-ports,然后执行in/run-zk conf，先查看bin/verify-default-ports

/usr/local/Cellar/apache-druid-0.20.0/bin> cat verify-default-ports
#!/usr/bin/env perl

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.

use strict;
use warnings;
use Socket;

sub try_bind {
  my ($port, $addr) = @_;

  socket(my $sock, PF_INET, SOCK_STREAM, Socket::IPPROTO_TCP) or die "socket: $!";
  setsockopt($sock, SOL_SOCKET, SO_REUSEADDR, pack("l", 1)) or die "setsockopt: $!";
  if (!bind($sock, sockaddr_in($port, $addr))) {
    print STDERR <<"EOT";
Cannot start up because port $port is already in use~~.

If you need to change your ports away from the defaults, check out the
configuration documentation:

  https://druid.apache.org/docs/latest/configuration/index.html

If you believe this check is in error, or if you have changed your ports away
from the defaults, you can skip this check using an environment variable:

  export DRUID_SKIP_PORT_CHECK=1

EOT
    exit 1;
  }
  shutdown($sock, 2);
}

my $skip_var = $ENV{'DRUID_SKIP_PORT_CHECK'};
if ($skip_var && $skip_var ne "0" && $skip_var ne "false" && $skip_var ne "f") {
  exit 0;
}

my @ports = @ARGV;
if (!@ports) {
  @ports = (2181, 1527, 8081, 8082, 8083, 8090, 8091, 8100, 8200, 8888);
}

for my $port (@ports) {
  try_bind($port, INADDR_ANY);
  try_bind($port, inet_aton("127.0.0.1"));
}

可以看到这里先扫描端口，对于它要使用的端口机型遍历，如果有人占用端口了，直接抛出错误，且不在执行后续的启动，这里吧@port里面的2181改成20182即可；然后再去run-zk conf查看

/usr/local/Cellar/apache-druid-0.20.0> cd bin
/usr/local/Cellar/apache-druid-0.20.0/bin> cat run-zk
#!/bin/bash -eu

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.

if [ "$#" -gt 1 ]
then
  >&2 echo "usage: $0 [conf-dir]"
  exit 1
fi

PWD="$(pwd)"
WHEREAMI="$(dirname "$0")"

if [ "$#" -lt 1 ] || [ "x$1" = "x" ]
then
  CONFDIR="$WHEREAMI"/../conf
else
  CONFDIR="$1"
fi

CONFDIR="$(cd "$CONFDIR" && pwd)/zk"
WHEREAMI="$(cd "$WHEREAMI" && pwd)"
JAVA_BIN="$(source "$WHEREAMI"/java-util && get_java_bin_dir)"
if [ -z "$JAVA_BIN" ]; then
  >&2 echo "Could not find java - please run $WHEREAMI/verify-java to confirm it is installed."
  exit 1
fi

cd "$WHEREAMI/.."
exec "$JAVA_BIN"/java `cat "$CONFDIR"/jvm.config | xargs` \
  -cp "$WHEREAMI/../lib/*:$CONFDIR" \
  -Dzookeeper.jmx.log4j.disable=true \
  org.apache.zookeeper.server.quorum.QuorumPeerMain \
  "$CONFDIR"/zoo.cfg

继续定位到加载了zoo.cfg文件

/usr/local/Cellar/apache-druid-0.20.0/conf/zk> cat zoo.cfg
#
# Server
#

tickTime=2000
dataDir=var/zk
clientPort=2181
initLimit=5
syncLimit=2

#
# Autopurge
#

autopurge.snapRetainCount=5
autopurge.purgeInterval=1

ok，找到2181端口的使用，修改clientPort=2182

方法2：直接暴力搜索使用2181的文件

如果使用方法2，需要在全项目范围内找，如果只是找conf文件夹，会遗漏端口校验的文件

/usr/local/Cellar/apache-druid-0.20.0> find ./ -type f | xargs grep -ri "2181"
.//bin/verify-default-ports:  @ports = (2181,1527, 8081, 8082, 8083, 8090, 8091, 8100, 8200, 8888);

.//var/sv/middleManager.log:2020-12-08T10:39:12,658 INFO [main-

...

.//conf/zk/zoo.cfg:clientPort=2181

显而易见，需要修改以上两个文件中的端口；修改完之后，/usr/local/Cellar/apache-druid-0.20.0/bin> ./start-micro-quickstart启动，http://localhost:8888/起来了；然后按照Tutorial的例子，直接加载Kafka的数据就好了

相关
  • Druid-quick-start：https://druid.apache.org/docs/latest/tutorials/index.html
  • Druid- Load streaming data from Apache Kafka：https://druid.apache.org/docs/latest/tutorials/tutorial-kafka.html
# zookeeper
Logo
Kafka开源项目指南

Kafka开源项目指南提供详尽教程，助开发者掌握其架构、配置和使用，实现高效数据流管理和实时处理。它高性能、可扩展，适合日志收集和实时数据处理，通过持久化保障数据安全，是企业大数据生态系统的核心。

更多推荐

Kafka入门（一） 概述、部署与API的简单使用

Kafka概述、部署与API的简单使用

avatar Kafka开源项目指南

基于canal和kafka同步，实现binlog同步ElasticSearch

文章目录前言elasticsearch 安装canal安装canal-adapter 安装及配置mysql 安装zk及kafaka安装查看效果注意事项前言中间件版本elasticsearch7.5.2canal1.1.4client-adapter1.1.5-alpha-1zookeeper3.4.13kafka2.6.0mysql5.7.31elasticsearch 安装{"settings"

avatar Kafka开源项目指南

基于 Iceberg 的湖仓一体架构在 B 站的实践

背景在B站，每天都有PB级的数据注入到大数据平台，经过离线或实时的ETL建模后，提供给下游的分析、推荐及预测等场景使用。面对如此大规模的数据，如何高效低成本地满足下游数据的分析需求，一直是我们重点的工作方向。我们之前的数据处理流程基本上是这样的：采集端将客户端埋点、服务端埋点、日志、业务数据库等数据收集到HDFS、Kafka等存储系统中，然后通过Hive、Spark、Fl...

avatar Kafka开源项目指南