go

Common Library for Go

Installation

go get -u github.com/common-library/go

Features

• ai
  • gemini
• archive
  • gzip
  • tar
  • zip
• aws
  • Amazon DynamoDB
  • Amazon S3
• command line
  • arguments
  • flag
• data structure
  • Deque
  • Queue
• database
  • dbmate
  • Elasticsearch v7/v8
  • MongoDB
  • Prometheus
  • Redis
  • ORM
    • beego
    • ent
    • GORM
    • sqlc
    • sqlx
  • SQL
    • Amazon DynamoDB
    • ClickHouse
    • Microsoft SQL Server
    • MySQL
    • Oracle
    • Postgres
    • SQLite
• event
  • cloudevents
• file
• grpc
• http
• json
• kubernetes
  • resource
    • client
    • custom resource
    • custom resource definition
• lock
• log
• long polling
• security
  • crypto
    • dsa
    • ecdsa
    • ed25519
    • rsa
• socket
• storage
  • MinIO
• utility

Test and Coverage

• prepare
  • Amazon
    • DynamoDB
      • docker run --name dynamodb --detach --publish 8000:8000 --env "-jar DynamoDBLocal.jar -sharedDb -inMemory" amazon/dynamodb-local:2.6.1
      • export DYNAMODB_URL=http://127.0.0.1:8000
    • S3
      • docker run --name s3mock --detach --publish 9090:9090 --publish 9191:9191 adobe/s3mock:4.1.1
      • export S3_URL=http://127.0.0.1:9090
  • dbmate
    • #curl -fsSL -o $HOME/.local/bin/dbmate https://github.com/amacneil/dbmate/releases/download/v2.24.2/dbmate-linux-amd64
    • chmod +x $HOME/.local/bin/dbmate
  • Elasticsearch
    • v7
      • docker run --name elasticsearch-v7 --detach --publish 19200:9200 --publish 19300:9300 --env discovery.type=single-node --env ES_JAVA_OPTS="-Xms500m -Xmx500m" elasticsearch:7.17.28
      • export ELASTICSEARCH_ADDRESS_V7=http://:19200
    • v8
      • docker network create elastic
      • docker run --name elasticsearch-v8 --net elastic --detach --publish 29200:9200 --publish 29300:9300 --env discovery.type=single-node --env ES_JAVA_OPTS="-Xms500m -Xmx500m" --env xpack.security.enabled=false elasticsearch:8.18.0
      • export ELASTICSEARCH_ADDRESS_V8=http://:29200
  • GEMINI
    • Set up API key
    • export GEMINI_API_KEY=${API_KEY}
  • MongoDB
    • docker run --name mongodb --detach --publish 27017:27017 mongo:8.0.9
    • export MONGODB_ADDRESS=:27017
  • Prometheus
    • wget https://github.com/prometheus/prometheus/releases/download/v3.3.1/prometheus-3.3.1.linux-amd64.tar.gz
    • tar xvzf prometheus-3.3.1.linux-amd64.tar.gz
    • cd prometheus-3.3.1.linux-amd64
    • yq -i '.scrape_configs[0].static_configs[0].targets = [":9095"]' prometheus.yml
    • ./prometheus --config.file=prometheus.yml --web.listen-address=:9095 &
    • export PROMETHEUS_ADDRESS=:9095
  • Redis
    • docker run --name redis --detach --publish 6379:6379 redis:7.4.3-alpine
    • export REDIS_ADDRESS=:6379
  • SQL
    • Amazon DynamoDB
      • docker run --name dynamodb --detach --publish 8000:8000 --env "-jar DynamoDBLocal.jar -sharedDb -inMemory" amazon/dynamodb-local:2.6.1
      • export DYNAMODB_URL=http://127.0.0.1:8000
    • ClickHouse
      • docker run --name clickhouse --detach --publish 19000:9000 --ulimit nofile=262144:262144 clickhouse/clickhouse-server:25.4.2.31-alpine
      • export CLICKHOUSE_DSN='clickhouse://default:@127.0.0.1:19000'
    • MySQL
      • docker run --name mysql --detach --publish 3306:3306 --env MYSQL_ROOT_PASSWORD=root mysql:9.3.0
      • export MYSQL_DSN='root:root@tcp(127.0.0.1)/'
    • PostgreSQL
      • docker run --name postgres --detach --publish 5432:5432 --env POSTGRES_PASSWORD=postgres postgres:17.4-alpine
      • export POSTGRESQL_DSN='host=localhost port=5432 user=postgres password=postgres sslmode=disable'
• Test
  • go clean -testcache && go test -cover ./...
• Coverage
  • make coverage file
    • go clean -testcache && go test -coverprofile=coverage.out -cover ./...
  • convert coverage file to html file
    • go tool cover -html=./coverage.out -o ./coverage.html

How to add a ent schema

• Assuming the schema name is Xxx
• go get entgo.io/ent/cmd/ent
• go run entgo.io/ent/cmd/ent new --target ./database/orm/ent/schema Xxx
• Modify ./database/orm/ent/schema/xxx.go
• go run entgo.io/ent/cmd/ent generate --feature sql/upsert ./database/orm/ent/schema

How to use sqlc

• Add or modify query file to ./database/orm/sqlc/queries
• Add or modify schema file to ./database/orm/sqlc/schema
• Modify ./database/orm/sqlc/sqlc.json
• go install github.com/sqlc-dev/sqlc/cmd/sqlc@v1.27.0
• sqlc generate --file ./database/orm/sqlc/sqlc.json

How to add grpc

• create protobuf IDL(Interface Definition Language) file
  • see grpc/sample/sample.proto
• convert IDL file to code
  • go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.31.0
  • go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.3.0
  • wget https://github.com/protocolbuffers/protobuf/releases/download/v3.20.3/protoc-3.20.3-linux-x86_64.zip
  • unzip protoc-3.20.3-linux-x86_64.zip -d protoc/
  • protoc/bin/protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative grpc/sample/sample.proto
• implement functions defined in IDL file
  • implement to satisfy implementServer interface
  • see grpc/sample/Server.go
  • see grpc/sample/Server_test.go