Every distribution inspected to extract out specific library versions and key dependencies.
This is helpful when you are tracking an open source security issue, want to know a version of a library to know if a feature is available, or to minimize dependency graph with your own java software that leverages kafka-client or kafka-streams libraries.
Additional cross-referencing with release notes and GitHub repositories to track differences between Confluent Community and Apache Kafka releases.
Process
Most of the information pulled into these documents is based on a script like the following:
for i in $(ls -dr kafka_*.tgz); do
gzcat $i | tar tfv - | grep "/NOTICE$"
gzcat $i | tar tfv - | (echo $i; grep -E 'libs/scala-library|libs/rocksdbjni-|libs/zstd-jni|libs/lz4-java|libs/snappy-java|libs/jackson-core|libs/slf4j-api|libs/slf4j-log4j|libs/zookeeper-[23]' | cut -d\/ -f2-; echo "")
done
for i in $(ls -dr confluent-community-*.tar); do
gzcat $i | tar tfv - | grep "/README$"
gzcat $i | tar tfv - | (echo $i; grep -E 'kafka/scala-library|kafka/rocksdbjni-|kafka/zstd-jni|kafka/lz4-java|kafka/snappy-java|kafka/jackson-core|libs/slf4j-api|kafka/slf4j-log4j|kafka/zookeeper-[23]|schema-registry/avro|schema-registry/protobuf' | cut -d\/ -f2-; echo "")
done
Addition effort is done inspecting release notes and GitHub (Apache’s and Confluent’s repositories).
Transparency
These scripts provide to showcase the process for our documentation; if you find an issue please let us know and we will update accordingly.