Enterprise-oriented Generic Proxy Solutions

Bytedance Inc.

Last update: Apr 19, 2023

Related tags

Network programming tls rust http proxy socks icap

Overview

G3 Project

中文版 README

About

This is the project we used to build enterprise-oriented generic proxy solutions, including but not limited to proxy / reverse proxy / load balancer / nat traversal.

Components

G3 Project is make up of many components.

The project level docs resides in the doc subdirectory, and you should see the links below for the important ones. Each component will have its own doc in its doc subdirectory.

g3proxy

A generic forward proxy solution, but you can also use it as tcp streaming / transparent proxy / reverse proxy as we have basic support built in.

See g3proxy for detailed introduction.

g3tiles

A work in progress reverse proxy solution.

g3bench

A benchmark tool for the test of g3proxy.

g3fcgen

Fake certificate generator for g3proxy.

Target Platform

Only Linux is supported currently. But we are open to PRs to add support to other platforms.

Dev-env Setup Guide

Follow dev-setup.

Standards

Follow standards.

Release and Packaging

We will set tags for each release of each component, in the form <name>-v<version>. You can use these tags to generate source tarballs. And we have added deb and rpm package files for each component that is ready for distribution.

If you want to do a release build:

generate a release tarball
```
./scripts/release/build_tarball.sh <name>-v<version>
```
All vendor sources will be added to the source tarball, so you can save the source tarball and build it offline at anywhere that have the compiler and dependencies installed.

build the package

For deb package:

tar xf <name>-<version>.tar.xz
cd <name>-<version>
./build_deb_from_tar.sh

For rpm package:

tar xvf <name>-<version>.tar.xz ./<name>-<version>/<name>.spec
cp <name>-<version>.tar.xz ~/rpmbuild/SOURCES/
rpmbuild -ba ./<name>-<version>/<name>.spec

If you want to build a package directly from the git repo:

For deb package:
```
./build_deb_from_git.sh <name>
```
For rpm package:
```
./build_rpm_from_git.sh <name>
```

Contribution

Please check Contributing for more details.

Code of Conduct

Please check Code of Conduct for more details.

Security

If you discover a potential security issue in this project, or think you may have discovered a security issue, we ask that you notify Bytedance Security via our security center or vulnerability reporting email.

Please do not create a public GitHub issue.

License

This project is licensed under the Apache-2.0 License.

Comments

build(deps): bump clap from 4.2.2 to 4.2.3
Bumps clap from 4.2.2 to 4.2.3.

Release notes

Sourced from clap's releases.

v4.2.3

[4.2.3] - 2023-04-18

Features

Command::styles for theming help/errors (behind unstable-styles)

Changelog

Sourced from clap's changelog.

[4.2.3] - 2023-04-18

Features

Command::styles for theming help/errors (behind unstable-styles)

Commits

b99c50f chore: Release

3f46258 docs: Update changelog

b0e0c59 Merge pull request #4843 from epage/style

cbea23e style: Make clippy happy

5ca3b2f refactor(help): Shrink binary size

ec7040e style: Make clippy happy

57974be refactor(help): Future proof styling

e10e2ad refactor(help): Split out separate usage style

8fd0a93 refactor(help): Clarify style meaning

7cf08e6 refactor(help): Remove unused hint

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies rust
opened by dependabot[bot] 0
build(deps): bump h2 from 0.3.17 to 0.3.18
Bumps h2 from 0.3.17 to 0.3.18.

Release notes

Sourced from h2's releases.

v0.3.18

What's Changed

fix: pending-accept remotely-reset streams pattern was checking is_local by @seanmonstar in hyperium/h2#676

Changelog

Sourced from h2's changelog.

0.3.18 (April 17, 2023)

Fix panic because of opposite check in is_remote_local().

Commits

1b9f070 v0.3.18

1c6fa28 fix: pending-accept remotely-reset streams pattern was checking is_local

See full diff in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies rust
opened by dependabot[bot] 0
build(deps): bump clap from 4.2.1 to 4.2.2
Bumps clap from 4.2.1 to 4.2.2.

Release notes

Sourced from clap's releases.

v4.2.2

[4.2.2] - 2023-04-13

Internal

Update dependencies

Changelog

Sourced from clap's changelog.

[4.2.2] - 2023-04-13

Internal

Update dependencies

Commits

6f40954 chore: Release

6d5394d docs: Update changelog

d7f742a Merge pull request #4832 from epage/update

9c85e1b chore: Update anstyle/anstream

0deb422 Merge pull request #4823 from epage/template

d34b51a Merge pull request #4821 from Bfault/typo

13d79b7 Fix typo

feddd12 style: Move away from banned fns

d3d45e8 chore(pre): Mirror exclude in pre-commit

c99ef98 chore(ci): Remove allow-print-in-tests due to MSRV

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies rust
opened by dependabot[bot] 0
build(deps): bump pyo3 from 0.18.2 to 0.18.3
Bumps pyo3 from 0.18.2 to 0.18.3.

Release notes

Sourced from pyo3's releases.

PyO3 0.18.3

This release fixes an issue with PyDateTime::type_object() incorrectly returning the type object for PyDate, which would cause some is_instance checks to behave incorrectly.

In addition, this release contains a handful of minor quality-of-life corrections over PyO3 0.18.2.

Thank you to the following users for the improvements:

@adamreichold @davidhewitt @decathorpe @ijl @mejrs @messense @sollyucko @willstott101 @wjones127

Changelog

Sourced from pyo3's changelog.

[0.18.3] - 2023-04-13

Added

Add GILProtected<T> to mediate concurrent access to a value using Python's global interpreter lock (GIL). #2975

Support PyASCIIObject / PyUnicode and associated methods on big-endian architectures. #3015

Add FFI definition _PyDict_Contains_KnownHash() for CPython 3.10 and up. #3088

Fixed

Fix compile error for #[pymethods] and #[pyfunction] called "output". #3022

Fix compile error in generated code for magic methods implemented as a #[staticmethod]. #3055

Fix is_instance for PyDateTime (would incorrectly check for a PyDate). #3071

Fix upstream deprecation of PyUnicode_InternImmortal since Python 3.10. #3071

Commits

5cdc133 release: 0.18.3

a8dd453 PEP 623: wstr, wstr_length, state changes

9ebdcb7 Define _PyDict_Contains_KnownHash() FFI for CPython 3.10 or later

7cc2644 Fix redundant imports in doc tests thereby fixing the careful CI job.

8498003 docs: show pattern for self.class.name in repr

63ff262 fix typo DateType -> DateTimeType, fixes #3069

48c0655 Add wrapper for PyASCIIObject state bitfield accesses based on bindgen

ce288e6 CI: Pin inventory to 0.3.4 in MSRV job

9dccad3 Emit a better error for bad argument names

24d2ddd Fix function name shadowing

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies rust
opened by dependabot[bot] 0
build(deps): bump h2 from 0.3.16 to 0.3.17
Bumps h2 from 0.3.16 to 0.3.17.

Release notes

Sourced from h2's releases.

v0.3.17

What's Changed

Add Error::is_library() method to check if the originated inside h2.

Add max_pending_accept_reset_streams(usize) option to client and server builders.

Fix theoretical memory growth when receiving too many HEADERS and then RST_STREAM frames faster than an application can accept them off the queue. (CVE-2023-26964)

Changelog

Sourced from h2's changelog.

0.3.17 (April 13, 2023)

Add Error::is_library() method to check if the originated inside h2.

Add max_pending_accept_reset_streams(usize) option to client and server builders.

Fix theoretical memory growth when receiving too many HEADERS and then RST_STREAM frames faster than an application can accept them off the queue. (CVE-2023-26964)

Commits

af4bcac v0.3.17

d3f37e9 feat: add max_pending_accept_reset_streams(n) options

5bc8e72 fix: limit the amount of pending-accept reset streams

8088ca6 feat: add Error::is_library method

481c31d chore: Use Cargo metadata for the MSRV build job

d3d50ef chore: Replace unmaintained/outdated GitHub Actions

45b9bcc chore: set rust-version in Cargo.toml (#664)

See full diff in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies rust
opened by dependabot[bot] 0
build(deps): bump serde_json from 1.0.95 to 1.0.96
Bumps serde_json from 1.0.95 to 1.0.96.

Release notes

Sourced from serde_json's releases.

v1.0.96

Guarantee that to_writer only writes valid UTF-8 strings (#1011, thanks @stepancheg)

Commits

187f7da Release 1.0.96

41199cc Merge pull request #1011 from stepancheg/utf-8

cd5ed82 Document to_writer only writes valid UTF-8 strings

ce53b86 Fix needless_borrow clippy lint in test

See full diff in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies rust
opened by dependabot[bot] 0
build(deps): bump capnp-rpc from 0.16.1 to 0.16.2
Bumps capnp-rpc from 0.16.1 to 0.16.2.

Commits

c79ad16 prepare for capnp-rpc-v0.16.2 release

1ab1ab6 tweak comment

daf6cfa expand comment on CapabilityServerSet to mention WeakCapabilityServerSet

80b95c1 Merge pull request #389 from canarysnort01/capabilityserverset-weak-refs

c62e3bc Add WeakCapabilityServerSet

27721b7 Merge pull request #387 from danielzfranklin/patch-1

8a7dff9 Improve getting started instructions

62e9cd5 Merge pull request #384 from marmeladema/fix-miri

d54b031 Ignore some tests under miri that are spawning a process because its not supp...

e31a5e4 Avoid some ptr-to/from-int casts

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies rust
opened by dependabot[bot] 0
build(deps): bump socket2 from 0.5.1 to 0.5.2
Bumps socket2 from 0.5.1 to 0.5.2.

Changelog

Sourced from socket2's changelog.

0.5.2

Add Unix socket methods to SockAddr (rust-lang/socket2#403 and rust-lang/socket2#429).

Added SockAddr::as_storage (rust-lang/socket2#417).

Added SockAddr::set_length (rust-lang/socket2#428).

Added Protocol::UDPLITE (rust-lang/socket2#427).

Update windows-sys to 0.48 (rust-lang/socket2#422).

Fixes Fuchsia target after it changes in 1.68, see https://github.com/rust-lang/rust/blob/master/RELEASES.md#version-1680-2023-03-09 (rust-lang/socket2#423).

Fixes musl target and adds it to the CI (rust-lang/socket2#426).

Commits

5bdcbf0 Release v0.5.2

bc1d8c0 Use more types from windows_sys crate

81d75df Add SockAddr::as_unix

3deba62 Cleanup PartialEq and Hash impls for SockAddr

c394ac3 Add SockAddr::set_length

6e48562 Move SockAddr::unix down

d9ffccc Don't install curl for FreeBSD CI

783b134 Update to FreeBSD 13.1 on Cirrus CI

c5c3101 Fix missing curl dependencies on Cirrus CI

f011ec4 Add Protocol::UDPLITE

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies rust
opened by dependabot[bot] 0
build(deps): bump serde from 1.0.159 to 1.0.160
Bumps serde from 1.0.159 to 1.0.160.

Release notes

Sourced from serde's releases.

v1.0.160

Make derived serializer/deserializer internals doc(hidden) (#2426, thanks @compiler-errors)

Commits

0c6a2bb Release 1.0.160

a80d830 Merge pull request #2426 from compiler-errors/dont-doc-private

5f3fd99 Make serializer/deserializer internals doc(hidden)

See full diff in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies rust
opened by dependabot[bot] 0
build(deps): bump webpki-roots from 0.22.6 to 0.23.0
Bumps webpki-roots from 0.22.6 to 0.23.0.

Commits

See full diff in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies rust
opened by dependabot[bot] 0