侧边栏壁纸
博主头像
问道

问道的小花园,总能给你带来惊喜

  • 累计撰写 42 篇文章
  • 累计创建 28 个标签
  • 累计收到 3 条评论

混合云资产管理项目(一)

问道
2023-01-24 / 0 评论 / 0 点赞 / 9 阅读 / 15,842 字 / 正在检测是否收录...
温馨提示:
本文最后更新于 2023-01-24,若内容或图片失效,请留言反馈。部分素材来自网络,若不小心影响到您的利益,请联系我们删除。

项目概述

资源统一检索

资源的属性抽象:

  • 所有资源的元数据信息<静态数据>(CMDB Object, 类似于k8s Runtime Object),
    • 资源Id
    • 资源创建时间
  • 资源的规格信息<动态数据>
    • 资源名称
    • 资源描述
    • IP
    • CPU
  • 资源的价格
    • 单价
    • 折扣
    • 购买信息
  • 资源的状态
    • 当前的状态
    • IP地址(借鉴了k8s的status)
  • 资源标签
    • resource_id tag_key tag_value
  • 资源关联关系
    • 资源嵌套, 一个主机 下面4个磁盘: resource_a resourceb , 基于关联表实现简单的图结构
  • 资源的额外信息:
    • 使用独立表存储
  1. 基于功能的CMDB, ECS, 支持删除,支持资源申请,...
  2. 偏向于云管,偏向于对资源的操作(BindEip, 关机,释放)
  3. 负责资源的全生命周期管理
    • 资源创建(同步)
    • 资源的更新
    • 资源的归档
    • 资源的销毁

脚手架生成代码

用一个脚手架工具来来生成初始化的代码,并且提供make工具来进行快捷操作。

脚手架工具代码仓库是infraboard/cmdb: cmdb (github.com)

安装mcube工具,用管理员工具

F:\go\project\0-shizhanxiangmu\myCMDB>go install github.com/infraboard/mcube/cmd/mcube@latest

#检查工具安装结果
C:\Users\zengz>mcube
mcube 分布式服务构建工具

Usage:
  mcube [flags]
  mcube [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  generate    代码生成器
  help        Help about any command
  project     项目初始化工具
  proto       项目protobuf管理类工具

Flags:
  -h, --help      help for mcube
  -v, --version   the mcube version

Use "mcube [command] --help" for more information about a command.

脚手架生成代码

F:\go\project\0-shizhanxiangmu\myCMDB>go install github.com/infraboard/mcube/cmd/mcube@latest

F:\go\project\0-shizhanxiangmu\myCMDB>mcube.exe project init
? 请输入项目包名称: (gitee.com/go-course/mcube-demo) gitee.com/wendao365/mycmdb

? 请输入项目包名称: gitee.com/wendao365/mycmdb
? 请输入项目描述:

X Sorry, your reply was invalid: Value is required
? 请输入项目描述: cmdb

? 请输入项目描述: cmdb
? 是否接入权限中心[keyauth] No
? 选择数据库类型: MySQL
? MySQL服务地址: (127.0.0.1:3306)

? MySQL服务地址: 127.0.0.1:3306
? 数据库名称: cmdb

? 数据库名称: cmdb
? 用户: root

? 用户: root
? 密码: *******
? 生成样例代码 Yes
? 选择HTTP框架: go-restful
项目初始化完成, 项目结构如下:
├───.gitignore (293b)
├───.mcube.yaml (191b)
├───.vscode
│       └───settings.json (233b)
├───README.md (4213b)
├───apps
│       ├───all
│       │       ├───api.go (138b)
│       │       ├───impl.go (169b)
│       │       └───internal.go (107b)
│       └───book
│               ├───api
│               │       ├───book.go (2250b)
│               │       └───http.go (2240b)
│               ├───app.go (2213b)
│               ├───impl
│               │       ├───book.go (3973b)
│               │       ├───dao.go (719b)
│               │       ├───impl.go (763b)
│               │       └───sql.go (325b)
│               └───pb
│                       └───book.proto (2350b)
├───client
│       ├───client.go (977b)
│       ├───client_test.go (628b)
│       └───config.go (164b)
├───cmd
│       ├───init.go (1202b)
│       ├───root.go (1239b)
│       └───start.go (3757b)
├───conf
│       ├───config.go (3885b)
│       ├───load.go (720b)
│       └───log.go (365b)
├───docs
│       ├───README.md (15b)
│       └───schema
│               └───tables.sql (849b)
├───etc
│       ├───config.env (476b)
│       ├───config.toml (313b)
│       └───unit_test.env (17b)
├───go.mod (33b)
├───main.go (90b)
├───makefile (2947b)
├───protocol
│       ├───grpc.go (1334b)
│       └───http.go (2907b)
├───swagger
│       └───docs.go (711b)
└───version
        └───version.go (630b)

F:\go\project\0-shizhanxiangmu\myCMDB>

后续处理

下载依赖

zengz@DESKTOP-Q3MJC54 MINGW64 /f/go/project/0-shizhanxiangmu/myCMDB/mycmdb
$ go mod tidy
go: finding module for package github.com/imdario/mergo
go: finding module for package github.com/infraboard/mcube/app
go: finding module for package github.com/go-playground/validator/v10  
go: finding module for package github.com/rs/xid
go: finding module for package github.com/emicklei/go-restful-openapi  
go: finding module for package github.com/infraboard/mcube/http/request
go: finding module for package github.com/infraboard/mcube/pb/request  
go: finding module for package github.com/emicklei/go-restful/v3       
go: downloading github.com/emicklei/go-restful/v3 v3.10.1
go: finding module for package github.com/infraboard/mcube/http/response
go: finding module for package github.com/infraboard/mcube/logger
go: finding module for package github.com/infraboard/mcube/logger/zap
go: finding module for package github.com/infraboard/mcube/exception
go: finding module for package github.com/infraboard/mcube/sqlbuilder
go: downloading github.com/infraboard/mcube v1.9.4
go: finding module for package google.golang.org/grpc
go: finding module for package github.com/infraboard/keyauth/client
go: finding module for package google.golang.org/grpc/credentials/insecure
go: finding module for package github.com/spf13/cobra
go: finding module for package github.com/BurntSushi/toml
go: finding module for package github.com/caarlos0/env/v6
go: finding module for package github.com/go-sql-driver/mysql
go: finding module for package github.com/emicklei/go-restful-openapi/v2
go: downloading github.com/go-sql-driver/mysql v1.7.0
go: finding module for package github.com/infraboard/mcube/grpc/middleware/recovery
go: downloading google.golang.org/grpc v1.51.0
go: finding module for package github.com/go-openapi/spec
go: finding module for package github.com/stretchr/testify/assert
go: found github.com/go-playground/validator/v10 in github.com/go-playground/validator/v10 v10.11.1
go: found github.com/imdario/mergo in github.com/imdario/mergo v0.3.13
go: found github.com/rs/xid in github.com/rs/xid v1.4.0
go: found github.com/emicklei/go-restful-openapi in github.com/emicklei/go-restful-openapi v1.4.1
go: found github.com/emicklei/go-restful/v3 in github.com/emicklei/go-restful/v3 v3.10.1
go: found google.golang.org/grpc in google.golang.org/grpc v1.51.0
go: found github.com/infraboard/keyauth/client in github.com/infraboard/keyauth v0.8.1
go: found google.golang.org/grpc/credentials/insecure in google.golang.org/grpc v1.51.0
go: found github.com/spf13/cobra in github.com/spf13/cobra v1.6.1
go: found github.com/BurntSushi/toml in github.com/BurntSushi/toml v1.2.1
go: found github.com/caarlos0/env/v6 in github.com/caarlos0/env/v6 v6.10.1
go: found github.com/go-sql-driver/mysql in github.com/go-sql-driver/mysql v1.7.0
go: found github.com/emicklei/go-restful-openapi/v2 in github.com/emicklei/go-restful-openapi/v2 v2.9.1
go: found github.com/go-openapi/spec in github.com/go-openapi/spec v0.20.7
go: found github.com/stretchr/testify/assert in github.com/stretchr/testify v1.8.1
go: downloading github.com/infraboard/mcube v1.8.11
go: downloading go.mongodb.org/mongo-driver v1.8.3

生成pb文件,移动文件会有错误,需要手动处理一下,

报错主要是Windows下会把路径拼错:

http://github.com)\infraboard | grep mcube$ ls F:\go\project\pkg\mod[github.com](
ls: cannot access 'F:goprojectpkgmodgithub.cominfraboard': No such file or directory

zengz@DESKTOP-Q3MJC54 MINGW64 /f/go/project/0-shizhanxiangmu/myCMDB/mycmdb
$ make pb
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
mkdir: created directory 'common'
mkdir: created directory 'common/pb'
mkdir: created directory 'common/pb/github.com'
mkdir: created directory 'common/pb/github.com/infraboard'
mkdir: created directory 'common/pb/github.com/infraboard/mcube'
mkdir: created directory 'common/pb/github.com/infraboard/mcube/pb'
cp: cannot stat 'F:goproject/pkg/mod/github.com/infraboard/mcube@v1.8.11/pb/*': No such file or directory
make: *** [makefile:64: pb] Error 1

查看用的什么版本的mcube

zengz@DESKTOP-Q3MJC54 MINGW64 /f/go/project/0-shizhanxiangmu/myCMDB/mycmdb (master)
$ go list -m "github.com/infraboard/mcube" | cut -d' ' -f2
v1.8.11

把对应目录下的pb文件拷过来,并删除go后缀文件

zengz@DESKTOP-Q3MJC54 MINGW64 /f/go/project/0-shizhanxiangmu/myCMDB/mycmdb (master)
$ ls /f/go/project/pkg/mod/github.com/infraboard | cut -d' ' -f2
keyauth@v0.6.4/
keyauth@v0.8.1/
mcenter@v0.0.7/
mcube@v1.5.4/
mcube@v1.8.11/
mcube@v1.9.0/
mcube@v1.9.1-0.20221116042658-6666a9ee4f42/
mcube@v1.9.2/
mcube@v1.9.3/
mcube@v1.9.3-0.20221130091016-310fcb7bb618/

zengz@DESKTOP-Q3MJC54 MINGW64 /f/go/project/0-shizhanxiangmu/myCMDB/mycmdb (master)
$ cp -r /f/go/project/pkg/mod/github.com/infraboard/mcube@v1.8.11/pb/* common/pb/github.com/infraboard/mcube/pb/

zengz@DESKTOP-Q3MJC54 MINGW64 /f/go/project/0-shizhanxiangmu/myCMDB/mycmdb (master)
$ rm -rf common/pb/github.com/infraboard/mcube/pb/*/*.go

zengz@DESKTOP-Q3MJC54 MINGW64 /f/go/project/0-shizhanxiangmu/myCMDB/mycmdb (master)
$ ls common/pb/github.com/infraboard/mcube/pb
example/  http/  page/  request/  resource/  response/


make install会出现问题,但cmd管理员又没有shell的命令

bash下make指令出现access denied

$ make install
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
go: downloading github.com/infraboard/mcube v1.9.4
go install github.com/infraboard/mcube/cmd/mcube: copying C:\Users\zengz\AppData\Local\Temp\go-build2884533382\b001\exe\a.out.exe: open C:\Program Files\Go\bin\mcube.exe: Access is den
ied.
make: *** [makefile:57: install] Error 1

根据make file的指令,在cmd的管理员下,直接执行试试,

install: ## Install depence go package
  @go install github.com/infraboard/mcube/cmd/mcube@latest
  @go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
  @go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
  @go install github.com/favadi/protoc-go-inject-tag@latest

单独运行命令

F:\go\project\0-shizhanxiangmu\myCMDB\mycmdb>go install github.com/infraboard/mcube/cmd/mcube@latest

F:\go\project\0-shizhanxiangmu\myCMDB\mycmdb>go install google.golang.org/protobuf/cmd/protoc-gen-go@latest

F:\go\project\0-shizhanxiangmu\myCMDB\mycmdb>go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

F:\go\project\0-shizhanxiangmu\myCMDB\mycmdb>go install github.com/favadi/protoc-go-inject-tag@latest

make gen生成代码

zengz@DESKTOP-Q3MJC54 MINGW64 /f/go/project/0-shizhanxiangmu/myCMDB/mycmdb (master)
$ make gen
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
main.go
apps\all\api.go
apps\all\impl.go
apps\all\internal.go
apps\book\impl\book.go
apps\book\impl\dao.go
apps\book\impl\impl.go
apps\book\impl\sql.go
client\client.go
client\config.go
cmd\init.go
cmd\root.go
cmd\start.go
conf\config.go
conf\load.go
conf\log.go
protocol\grpc.go
protocol\http.go
swagger\docs.go

数据库设计

在mysql中运行下面脚本,建表。

SET NAMES utf8mb4;
SET FOREIGN_KEY_CHECKS = 0;

-- ----------------------------
-- Table structure for resource_cost
-- ----------------------------
DROP TABLE IF EXISTS `resource_cost`;
CREATE TABLE `resource_cost`  (
  `resource_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `pay_mode` tinyint(1) NULL DEFAULT NULL COMMENT '资源付费方式',
  `pay_mode_detail` text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '付费详情',
  `sale_price` int(0) NOT NULL COMMENT '官方价(分)',
  `real_cost` int(0) NOT NULL COMMENT '实践费用(分)',
  `policy` double(255, 0) NOT NULL COMMENT '折扣',
  `unit_price` int(0) NOT NULL COMMENT '单价(分)',
  PRIMARY KEY (`resource_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for resource_host
-- ----------------------------
DROP TABLE IF EXISTS `resource_host`;
CREATE TABLE `resource_host`  (
  `resource_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `gpu_spec` text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `os_type` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `os_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `image_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `internet_max_bandwidth_out` int(0) NOT NULL,
  `internet_max_bandwidth_in` int(0) NOT NULL,
  PRIMARY KEY (`resource_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for resource_meta
-- ----------------------------
DROP TABLE IF EXISTS `resource_meta`;
CREATE TABLE `resource_meta`  (
  `resource_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源ID',
  `domain` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源所属域',
  `namespace` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源所属空间',
  `env` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源所属环境',
  `sync_at` int(0) NOT NULL COMMENT '同步时间',
  `credential_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源管理的API 凭证',
  `create_at` int(0) NOT NULL COMMENT '创建时间',
  `serial_number` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '序列号',
  PRIMARY KEY (`resource_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for resource_relation
-- ----------------------------
DROP TABLE IF EXISTS `resource_relation`;
CREATE TABLE `resource_relation`  (
  `id` int(0) UNSIGNED NOT NULL AUTO_INCREMENT COMMENT '关系Id',
  `src` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源Id',
  `target` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '关联的资源Id',
  PRIMARY KEY (`id`) USING BTREE,
  UNIQUE INDEX `_idx_id`(`src`, `target`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for resource_spec
-- ----------------------------
DROP TABLE IF EXISTS `resource_spec`;
CREATE TABLE `resource_spec`  (
  `resource_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源Id',
  `vendor` tinyint(1) NOT NULL COMMENT '厂商类型',
  `resource_type` tinyint(1) NOT NULL COMMENT '资源类型',
  `region` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源地域',
  `zone` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源区域',
  `owner` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源所属账号',
  `name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源名称',
  `category` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源种类',
  `type` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '规格',
  `description` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源描述',
  `expire_at` int(0) NOT NULL COMMENT '过期时间',
  `update_at` int(0) NOT NULL COMMENT '更新时间',
  `release_protection` tinyint(1) NULL DEFAULT NULL COMMENT '是否包含',
  `cpu` int(0) NOT NULL,
  `gpu` int(0) NOT NULL,
  `memory` int(0) NOT NULL,
  `storage` int(0) NOT NULL,
  `band_width` int(0) NOT NULL,
  PRIMARY KEY (`resource_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for resource_status
-- ----------------------------
DROP TABLE IF EXISTS `resource_status`;
CREATE TABLE `resource_status`  (
  `resource_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源Id',
  `phase` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '状态',
  `lock_mode` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '锁定模式',
  `lock_reason` text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '原因',
  `public_ip` text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '公网Ip, 以逗号分隔',
  `private_ip` text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '内网Ip, 以逗号分隔',
  PRIMARY KEY (`resource_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for resource_tag
-- ----------------------------
DROP TABLE IF EXISTS `resource_tag`;
CREATE TABLE `resource_tag`  (
  `resource_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '资源Id',
  `purpose` tinyint(0) NOT NULL COMMENT '标签用途',
  `key` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '标签的Key',
  `value` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '标签的Value',
  `describe` text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '标签描述',
  `weight` int(0) NOT NULL COMMENT '标签权重',
  `read_only` tinyint(0) NOT NULL COMMENT '是否只读',
  `hidden` tinyint(0) NOT NULL COMMENT '是否隐藏',
  UNIQUE INDEX `_idx_id`(`resource_id`, `key`, `value`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for secret
-- ----------------------------
DROP TABLE IF EXISTS `secret`;
CREATE TABLE `secret`  (
  `id` varchar(64) CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
  `create_at` bigint(0) NOT NULL,
  `description` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `vendor` tinyint(1) NOT NULL,
  `address` varchar(255) CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
  `allow_regions` text CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
  `crendential_type` tinyint(1) NOT NULL,
  `api_key` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_croatian_ci NOT NULL,
  `api_secret` text CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
  `request_rate` int(0) NOT NULL,
  `domain` varchar(255) CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
  `namespace` varchar(255) CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL,
  PRIMARY KEY (`id`) USING BTREE,
  UNIQUE INDEX `idx_key`(`api_key`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = latin1 COLLATE = latin1_swedish_ci ROW_FORMAT = Dynamic;

0

评论区