项目概述
资源统一检索
资源的属性抽象:
- 所有资源的元数据信息<静态数据>(CMDB Object, 类似于k8s Runtime Object),
- 资源Id
- 资源创建时间
- 资源的规格信息<动态数据>
- 资源名称
- 资源描述
- IP
- CPU
- 资源的价格
- 单价
- 折扣
- 购买信息
- 资源的状态
- 当前的状态
- IP地址(借鉴了k8s的status)
- 资源标签
- resource_id tag_key tag_value
- 资源关联关系
- 资源嵌套， 一个主机 下面4个磁盘: resource_a resourceb , 基于关联表实现简单的图结构
- 资源的额外信息:
- 使用独立表存储
- 基于功能的CMDB, ECS, 支持删除，支持资源申请，...
- 偏向于云管，偏向于对资源的操作(BindEip, 关机，释放)
- 负责资源的全生命周期管理
- 资源创建(同步)
- 资源的更新
- 资源的归档
- 资源的销毁
脚手架生成代码
用一个脚手架工具来来生成初始化的代码，并且提供make工具来进行快捷操作。
脚手架工具代码仓库是infraboard/cmdb: cmdb (github.com)
安装mcube工具，用管理员工具
F:\go\project\0-shizhanxiangmu\myCMDB>go install github.com/infraboard/mcube/cmd/mcube@latest
#检查工具安装结果
C:\Users\zengz>mcube
mcube 分布式服务构建工具
Usage:
mcube [flags]
mcube [command]
Available Commands:
completion Generate the autocompletion script for the specified shell
generate 代码生成器
help Help about any command
project 项目初始化工具
proto 项目protobuf管理类工具
Flags:
-h, --help help for mcube
-v, --version the mcube version
Use "mcube [command] --help" for more information about a command.
脚手架生成代码
F:\go\project\0-shizhanxiangmu\myCMDB>go install github.com/infraboard/mcube/cmd/mcube@latest
F:\go\project\0-shizhanxiangmu\myCMDB>mcube.exe project init
? 请输入项目包名称: (gitee.com/go-course/mcube-demo) gitee.com/wendao365/mycmdb
? 请输入项目包名称: gitee.com/wendao365/mycmdb
? 请输入项目描述:
X Sorry, your reply was invalid: Value is required
? 请输入项目描述: cmdb
? 请输入项目描述: cmdb
? 是否接入权限中心[keyauth] No
? 选择数据库类型: MySQL
? MySQL服务地址: (127.0.0.1:3306)
? MySQL服务地址: 127.0.0.1:3306
? 数据库名称: cmdb
? 数据库名称: cmdb
? 用户: root
? 用户: root
? 密码: *******
? 生成样例代码 Yes
? 选择HTTP框架: go-restful
项目初始化完成, 项目结构如下:
├───.gitignore (293b)
├───.mcube.yaml (191b)
├───.vscode
│ └───settings.json (233b)
├───README.md (4213b)
├───apps
│ ├───all
│ │ ├───api.go (138b)
│ │ ├───impl.go (169b)
│ │ └───internal.go (107b)
│ └───book
│ ├───api
│ │ ├───book.go (2250b)
│ │ └───http.go (2240b)
│ ├───app.go (2213b)
│ ├───impl
│ │ ├───book.go (3973b)
│ │ ├───dao.go (719b)
│ │ ├───impl.go (763b)
│ │ └───sql.go (325b)
│ └───pb
│ └───book.proto (2350b)
├───client
│ ├───client.go (977b)
│ ├───client_test.go (628b)
│ └───config.go (164b)
├───cmd
│ ├───init.go (1202b)
│ ├───root.go (1239b)
│ └───start.go (3757b)
├───conf
│ ├───config.go (3885b)
│ ├───load.go (720b)
│ └───log.go (365b)
├───docs
│ ├───README.md (15b)
│ └───schema
│ └───tables.sql (849b)
├───etc
│ ├───config.env (476b)
│ ├───config.toml (313b)
│ └───unit_test.env (17b)
├───go.mod (33b)
├───main.go (90b)
├───makefile (2947b)
├───protocol
│ ├───grpc.go (1334b)
│ └───http.go (2907b)
├───swagger
│ └───docs.go (711b)
└───version
└───version.go (630b)
F:\go\project\0-shizhanxiangmu\myCMDB>
后续处理
下载依赖
zengz@DESKTOP-Q3MJC54 MINGW64 /f/go/project/0-shizhanxiangmu/myCMDB/mycmdb
$ go mod tidy
go: finding module for package github.com/imdario/mergo
go: finding module for package github.com/infraboard/mcube/app
go: finding module for package github.com/go-playground/validator/v10
go: finding module for package github.com/rs/xid
go: finding module for package github.com/emicklei/go-restful-openapi
go: finding module for package github.com/infraboard/mcube/http/request
go: finding module for package github.com/infraboard/mcube/pb/request
go: finding module for package github.com/emicklei/go-restful/v3
go: downloading github.com/emicklei/go-restful/v3 v3.10.1
go: finding module for package github.com/infraboard/mcube/http/response
go: finding module for package github.com/infraboard/mcube/logger
go: finding module for package github.com/infraboard/mcube/logger/zap
go: finding module for package github.com/infraboard/mcube/exception
go: finding module for package github.com/infraboard/mcube/sqlbuilder
go: downloading github.com/infraboard/mcube v1.9.4
go: finding module for package google.golang.org/grpc
go: finding module for package github.com/infraboard/keyauth/client
go: finding module for package google.golang.org/grpc/credentials/insecure
go: finding module for package github.com/spf13/cobra
go: finding module for package github.com/BurntSushi/toml
go: finding module for package github.com/caarlos0/env/v6
go: finding module for package github.com/go-sql-driver/mysql
go: finding module for package github.com/emicklei/go-restful-openapi/v2
go: downloading github.com/go-sql-driver/mysql v1.7.0
go: finding module for package github.com/infraboard/mcube/grpc/middleware/recovery
go: downloading google.golang.org/grpc v1.51.0
go: finding module for package github.com/go-openapi/spec
go: finding module for package github.com/stretchr/testify/assert
go: found github.com/go-playground/validator/v10 in github.com/go-playground/validator/v10 v10.11.1
go: found github.com/imdario/mergo in github.com/imdario/mergo v0.3.13
go: found github.com/rs/xid in github.com/rs/xid v1.4.0
go: found github.com/emicklei/go-restful-openapi in github.com/emicklei/go-restful-openapi v1.4.1
go: found github.com/emicklei/go-restful/v3 in github.com/emicklei/go-restful/v3 v3.10.1
go: found google.golang.org/grpc in google.golang.org/grpc v1.51.0
go: found github.com/infraboard/keyauth/client in github.com/infraboard/keyauth v0.8.1
go: found google.golang.org/grpc/credentials/insecure in google.golang.org/grpc v1.51.0
go: found github.com/spf13/cobra in github.com/spf13/cobra v1.6.1
go: found github.com/BurntSushi/toml in github.com/BurntSushi/toml v1.2.1
go: found github.com/caarlos0/env/v6 in github.com/caarlos0/env/v6 v6.10.1
go: found github.com/go-sql-driver/mysql in github.com/go-sql-driver/mysql v1.7.0
go: found github.com/emicklei/go-restful-openapi/v2 in github.com/emicklei/go-restful-openapi/v2 v2.9.1
go: found github.com/go-openapi/spec in github.com/go-openapi/spec v0.20.7
go: found github.com/stretchr/testify/assert in github.com/stretchr/testify v1.8.1
go: downloading github.com/infraboard/mcube v1.8.11
go: downloading go.mongodb.org/mongo-driver v1.8.3
生成pb文件，移动文件会有错误，需要手动处理一下，
报错主要是Windows下会把路径拼错:
http://github.com)\infraboard | grep mcube$ ls F:\go\project\pkg\mod[github.com](
ls: cannot access 'F:goprojectpkgmodgithub.cominfraboard': No such file or directory
zengz@DESKTOP-Q3MJC54 MINGW64 /f/go/project/0-shizhanxiangmu/myCMDB/mycmdb
$ make pb
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
mkdir: created directory 'common'
mkdir: created directory 'common/pb'
mkdir: created directory 'common/pb/github.com'
mkdir: created directory 'common/pb/github.com/infraboard'
mkdir: created directory 'common/pb/github.com/infraboard/mcube'
mkdir: created directory 'common/pb/github.com/infraboard/mcube/pb'
cp: cannot stat 'F:goproject/pkg/mod/github.com/infraboard/mcube@v1.8.11/pb/*': No such file or directory
make: *** [makefile:64: pb] Error 1
查看用的什么版本的mcube
zengz@DESKTOP-Q3MJC54 MINGW64 /f/go/project/0-shizhanxiangmu/myCMDB/mycmdb (master)
$ go list -m "github.com/infraboard/mcube" | cut -d' ' -f2
v1.8.11
把对应目录下的pb文件拷过来，并删除go后缀文件
zengz@DESKTOP-Q3MJC54 MINGW64 /f/go/project/0-shizhanxiangmu/myCMDB/mycmdb (master)
$ ls /f/go/project/pkg/mod/github.com/infraboard | cut -d' ' -f2
keyauth@v0.6.4/
keyauth@v0.8.1/
mcenter@v0.0.7/
mcube@v1.5.4/
mcube@v1.8.11/
mcube@v1.9.0/
mcube@v1.9.1-0.20221116042658-6666a9ee4f42/
mcube@v1.9.2/
mcube@v1.9.3/
mcube@v1.9.3-0.20221130091016-310fcb7bb618/
zengz@DESKTOP-Q3MJC54 MINGW64 /f/go/project/0-shizhanxiangmu/myCMDB/mycmdb (master)
$ cp -r /f/go/project/pkg/mod/github.com/infraboard/mcube@v1.8.11/pb/* common/pb/github.com/infraboard/mcube/pb/
zengz@DESKTOP-Q3MJC54 MINGW64 /f/go/project/0-shizhanxiangmu/myCMDB/mycmdb (master)
$ rm -rf common/pb/github.com/infraboard/mcube/pb/*/*.go
zengz@DESKTOP-Q3MJC54 MINGW64 /f/go/project/0-shizhanxiangmu/myCMDB/mycmdb (master)
$ ls common/pb/github.com/infraboard/mcube/pb
example/ http/ page/ request/ resource/ response/
make install会出现问题，但cmd管理员又没有shell的命令
bash下make指令出现access denied
$ make install
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
go: downloading github.com/infraboard/mcube v1.9.4
go install github.com/infraboard/mcube/cmd/mcube: copying C:\Users\zengz\AppData\Local\Temp\go-build2884533382\b001\exe\a.out.exe: open C:\Program Files\Go\bin\mcube.exe: Access is den
ied.
make: *** [makefile:57: install] Error 1
根据make file的指令，在cmd的管理员下，直接执行试试，
install: ## Install depence go package
@go install github.com/infraboard/mcube/cmd/mcube@latest
@go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
@go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
@go install github.com/favadi/protoc-go-inject-tag@latest
单独运行命令
F:\go\project\0-shizhanxiangmu\myCMDB\mycmdb>go install github.com/infraboard/mcube/cmd/mcube@latest
F:\go\project\0-shizhanxiangmu\myCMDB\mycmdb>go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
F:\go\project\0-shizhanxiangmu\myCMDB\mycmdb>go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
F:\go\project\0-shizhanxiangmu\myCMDB\mycmdb>go install github.com/favadi/protoc-go-inject-tag@latest
make gen生成代码
zengz@DESKTOP-Q3MJC54 MINGW64 /f/go/project/0-shizhanxiangmu/myCMDB/mycmdb (master)
$ make gen
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
main.go
apps\all\api.go
apps\all\impl.go
apps\all\internal.go
apps\book\impl\book.go
apps\book\impl\dao.go
apps\book\impl\impl.go
apps\book\impl\sql.go
client\client.go
client\config.go
cmd\init.go
cmd\root.go
cmd\start.go
conf\config.go
conf\load.go
conf\log.go
protocol\grpc.go
protocol\http.go
swagger\docs.go
数据库设计
在mysql中运行下面脚本，建表。
SET NAMES utf8mb4;
SET FOREIGN_KEY_CHECKS = 0;
-- ----------------------------
-- Table structure for resource_cost
-- ----------------------------
DROP TABLE IF EXISTS `resource_cost`;
CREATE TABLE `resource_cost` (
`resource_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
`pay_mode` tinyint(1) NULL DEFAULT NULL COMMENT '资源付费方式',
`pay_mode_detail` text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '付费详情',
`sale_price` int(0) NOT NULL COMMENT '官方价(分)',
`real_cost` int(0) NOT NULL COMMENT '实践费用(分)',
`policy` double(255, 0) NOT NULL COMMENT '折扣',
`unit_price` int(0) NOT NULL COMMENT '单价(分)',
PRIMARY KEY (`resource_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;
-- ----------------------------
-- Table structure for resource_host
-- ----------------------------
DROP TABLE IF EXISTS `resource_host`;
CREATE TABLE `resource_host` (
`resource_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
`gpu_spec` text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
`os_type` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
`os_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
`image_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
`internet_max_bandwidth_out` int(0) NOT NULL,
`internet_max_bandwidth_in` int(0) NOT NULL,
PRIMARY KEY (`resource_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;
-- ----------------------------
-- Table structure for resource_meta
-- ----------------------------
DROP TABLE IF EXISTS `resource_meta`;
CREATE TABLE `resource_meta` (
`resource_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源ID',
`domain` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源所属域',
`namespace` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源所属空间',
`env` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源所属环境',
`sync_at` int(0) NOT NULL COMMENT '同步时间',
`credential_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源管理的API 凭证',
`create_at` int(0) NOT NULL COMMENT '创建时间',
`serial_number` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '序列号',
PRIMARY KEY (`resource_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;
-- ----------------------------
-- Table structure for resource_relation
-- ----------------------------
DROP TABLE IF EXISTS `resource_relation`;
CREATE TABLE `resource_relation` (
`id` int(0) UNSIGNED NOT NULL AUTO_INCREMENT COMMENT '关系Id',
`src` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源Id',
`target` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '关联的资源Id',
PRIMARY KEY (`id`) USING BTREE,
UNIQUE INDEX `_idx_id`(`src`, `target`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;
-- ----------------------------
-- Table structure for resource_spec
-- ----------------------------
DROP TABLE IF EXISTS `resource_spec`;
CREATE TABLE `resource_spec` (
`resource_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源Id',
`vendor` tinyint(1) NOT NULL COMMENT '厂商类型',
`resource_type` tinyint(1) NOT NULL COMMENT '资源类型',
`region` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源地域',
`zone` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源区域',
`owner` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源所属账号',
`name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源名称',
`category` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源种类',
`type` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '规格',
`description` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源描述',
`expire_at` int(0) NOT NULL COMMENT '过期时间',
`update_at` int(0) NOT NULL COMMENT '更新时间',
`release_protection` tinyint(1) NULL DEFAULT NULL COMMENT '是否包含',
`cpu` int(0) NOT NULL,
`gpu` int(0) NOT NULL,
`memory` int(0) NOT NULL,
`storage` int(0) NOT NULL,
`band_width` int(0) NOT NULL,
PRIMARY KEY (`resource_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;
-- ----------------------------
-- Table structure for resource_status
-- ----------------------------
DROP TABLE IF EXISTS `resource_status`;
CREATE TABLE `resource_status` (
`resource_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源Id',
`phase` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '状态',
`lock_mode` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '锁定模式',
`lock_reason` text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '原因',
`public_ip` text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '公网Ip, 以逗号分隔',
`private_ip` text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '内网Ip， 以逗号分隔',
PRIMARY KEY (`resource_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;
-- ----------------------------
-- Table structure for resource_tag
-- ----------------------------
DROP TABLE IF EXISTS `resource_tag`;
CREATE TABLE `resource_tag` (
`resource_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源Id',
`purpose` tinyint(0) NOT NULL COMMENT '标签用途',
`key` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '标签的Key',
`value` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '标签的Value',
`describe` text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '标签描述',
`weight` int(0) NOT NULL COMMENT '标签权重',
`read_only` tinyint(0) NOT NULL COMMENT '是否只读',
`hidden` tinyint(0) NOT NULL COMMENT '是否隐藏',
UNIQUE INDEX `_idx_id`(`resource_id`, `key`, `value`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;
-- ----------------------------
-- Table structure for secret
-- ----------------------------
DROP TABLE IF EXISTS `secret`;
CREATE TABLE `secret` (
`id` varchar(64) CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
`create_at` bigint(0) NOT NULL,
`description` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
`vendor` tinyint(1) NOT NULL,
`address` varchar(255) CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
`allow_regions` text CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
`crendential_type` tinyint(1) NOT NULL,
`api_key` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_croatian_ci NOT NULL,
`api_secret` text CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
`request_rate` int(0) NOT NULL,
`domain` varchar(255) CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
`namespace` varchar(255) CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
PRIMARY KEY (`id`) USING BTREE,
UNIQUE INDEX `idx_key`(`api_key`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = latin1 COLLATE = latin1_swedish_ci ROW_FORMAT = Dynamic;
评论区