文件历史

提交图

33 次代码提交

作者 SHA1 备注 提交日期
Asim Aslam cae6fbbe76 Framework hardening: security, reliability, and developer experience improvements (#2826)
* fix: remove deprecated rand.Seed calls

Go 1.20+ automatically seeds the global random number generator.
These calls are no-ops and generate warnings with newer Go versions.

Removed from:
- selector/strategy.go
- registry/cache/cache.go
- broker/memory.go
- broker/http.go
- cmd/cmd.go
- transport/memory.go

Co-authored-by: Shelley <shelley@exe.dev>

* fix: handle previously ignored errors

- MySQL store: properly handle prepared statement errors in initDB()
- Consul registry: handle client creation errors in Client() method

These silent failures could cause hard-to-debug issues in production.

Co-authored-by: Shelley <shelley@exe.dev>

* feat(genai): improve provider interface with context and streaming

Breaking changes:
- Generate() and Stream() now require context.Context as first parameter
- Stream.Close() added for proper resource cleanup

Improvements:
- Proper context support for cancellation and timeouts
- Real SSE streaming for OpenAI and Gemini text generation
- Better error handling with wrapped errors and API error responses
- Thread-safe provider registry with sync.RWMutex
- New options: WithMaxTokens, WithTemperature, WithTimeout
- Stream has proper Close() method for cleanup
- Results can include Error field for per-chunk errors

Provider updates:
- OpenAI: true streaming with SSE parsing, proper HTTP client with timeout
- Gemini: true streaming with streamGenerateContent endpoint
- Default model updated to gpt-4o-mini (OpenAI) and gemini-2.0-flash (Gemini)

Co-authored-by: Shelley <shelley@exe.dev>

* feat(tls): make TLS secure by default, configurable via environment

BREAKING: TLS now verifies certificates by default. Set MICRO_TLS_INSECURE=true
to restore previous behavior (NOT recommended for production).

Changes:
- Add util/tls.Config(), SecureConfig(), InsecureConfig(), ConfigFromEnv() helpers
- Update all components to use ConfigFromEnv() instead of hardcoded InsecureSkipVerify
- Set MinVersion to TLS 1.2 for all TLS configs

Affected components:
- broker/http
- broker/rabbitmq
- registry/etcd
- registry/consul
- transport/grpc

This improves security posture while allowing opt-out for development environments.

Co-authored-by: Shelley <shelley@exe.dev>

* feat(tls): add TLS helpers with opt-in secure mode

NOT a breaking change - keeps InsecureSkipVerify=true as default for
local development compatibility.

New util/tls helpers:
- Config() - returns config based on MICRO_TLS_SECURE env var
- SecureConfig() - certificate verification enabled
- InsecureConfig() - certificate verification disabled (dev only)

For production security, use one of:
- Set MICRO_TLS_SECURE=true with proper CA-signed certs
- Use a service mesh (Istio, Linkerd) for automatic mTLS
- Configure TLSConfig directly with your certificates

Also: Changed CLI alias from 'g' to 'gen' for clarity
- micro generate handler -> micro gen handler

Co-authored-by: Shelley <shelley@exe.dev>

* refactor(cli): rename generate directory to gen for consistency

Directory name now matches the command alias:
  cmd/micro/cli/gen/ -> micro gen handler

Co-authored-by: Shelley <shelley@exe.dev>

---------

Co-authored-by: Shelley <shelley@exe.dev>
2026-01-27 10:39:25 +00:00
Copilot baeb282cf1 Enhance cache rate limiting to prevent registry overload during rolling deployments (#2824)
* Initial plan

* Enhance cache rate limiting to protect against rolling deployment scenarios

Per @asim's feedback, rate limiting now applies to ALL cache refresh attempts
(not just error cases) to prevent registry overload during rolling deployments.

Changes:
- Rate limit ALL refresh attempts using lastRefreshAttempt tracking
- Always return stale cache if available (even if expired) when rate limiting
- Only return ErrNotFound when no cache exists during rate limit period
- Rate limiting happens inside singleflight to avoid race conditions
- Update test to reflect new behavior (no retry when stale cache + rate limit)
- Enhanced documentation with rolling deployment scenario examples

This addresses the scenario where all upstream services expire their cache
simultaneously during a downstream rolling deployment, which would previously
cause a stampede to the registry under high QPS.

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Remove unused failedAttempts and consecutiveFailures fields

Per @asim's feedback, these fields are no longer needed since the new
rate limiting strategy uses lastRefreshAttempt to track ALL refresh
attempts, not just failed ones.

Removed:
- failedAttempts map[string]time.Time
- consecutiveFailures int
- All code that sets these fields

The rate limiting logic now only uses lastRefreshAttempt, making these
fields redundant. All tests continue to pass.

Co-authored-by: asim <17530+asim@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-01-14 10:57:14 +00:00
Copilot 2869cc16d1 Add adaptive throttling to registry cache to prevent penetration when no stale cache exists (#2823)
* Initial plan

* Add adaptive throttling to prevent cache penetration without stale cache

Implement rate limiting mechanism to protect etcd when:
- Cache is empty (no stale data available)
- Registry is failing/overloaded
- Multiple sequential requests arrive

Changes:
- Add MinimumRetryInterval option (default 5s)
- Track last failed attempt time per service
- Throttle requests within retry interval when no cache exists
- Clear throttling on successful lookup
- Add comprehensive tests validating throttling behavior

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Fix WaitGroup race condition in events stream tests

The WaitGroup.Add() calls were happening after goroutines started,
causing a race where Done() could be called before Add(), resulting
in "negative WaitGroup counter" panics.

Fixes:
- Move wg.Add(1) before goroutine in TestConsumeTopic
- Move wg.Add(2) before first goroutine in TestConsumeGroup

Co-authored-by: asim <17530+asim@users.noreply.github.com>

* Update registry cache documentation with adaptive throttling feature

Document the new adaptive throttling feature that prevents cache
penetration when registry is failing and no stale cache exists.

Includes:
- Feature overview
- Usage examples with configuration options
- Explanation of throttling behavior
- Example scenario demonstrating protection

Co-authored-by: asim <17530+asim@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
2026-01-09 12:27:55 +00:00
Roman Perekhod ad73add529 fix the concurrent map writes error (#2794) 2025-07-16 07:27:45 +01:00
Jörn Friedrich Dreyer e032a6aafd update ttl for updated nodes only (#2740)
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2024-11-15 14:08:17 +00:00
Jörn Friedrich Dreyer 14a1791011 invalidate service if node was not updated (#2736)
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2024-10-28 15:34:38 +00:00
littlecoderonway 4d5b3b084f Fix registry cache broken (#2715)
* services is shared between routines, when application change services returned by get function may lead to other routine set a changed services to cache

* services is shared between routines, when application change services returned by get function may lead to other routine set a changed services to cache

---------

Co-authored-by: Shuaihu Liu <shuaihu.liu@shopee.com>
2024-07-02 09:30:51 +01:00
asim 610c00859f v5 2024-06-04 21:40:43 +01:00
Lukasz Raczylo a7522e7d6c fix: struct field alignment (#2632) 2023-04-26 02:16:34 +02:00
David Brouwer a3980c2308 feat: add test framework & refactor RPC server (#2579)
Co-authored-by: Rene Jochum <rene@jochum.dev>
2022-10-20 13:00:50 +02:00
keepstep c25dee7c8a fix(registry/cache): do not watch when ttl=0 eg: some custom registry no s… (#2580)
Co-authored-by: keep <keep@pa.com>
2022-10-08 15:19:48 +02:00
Rene Jochum 010b1d9f11 fix: linting issues (#2566) 2022-09-30 20:32:55 +02:00
David Brouwer 85c0b0b8eb fix: some linting issues (#2563) 2022-09-30 16:27:07 +02:00
Mohamed MHAMDI 1db36357d5 feat(logger): add logger option to all micro components (override DefaultLogger) closes #2556 (#2559)
Run tests / Test repo (push) Has been cancelled
* feat(logger): add logger option to all components

* fix: refactor api/rpc.go

* fix: refactor api/stream.go

* fix: api/options.go comment

* fix(logger): do not use logger.Helper internally

* fix(logger): fix comments

* fix(logger): use level.Enabled method

* fix: rename mlogger to log

* fix: run go fmt

* fix: log level

* fix: factories

Co-authored-by: Mohamed MHAMDI <mmhamdi@hubside.com>
Co-authored-by: Davincible <david.brouwer.99@gmail.com>
2022-09-29 16:44:53 +02:00
JellyTony 65a2e4ed02 fix(registry): fix watch services only first time (#2517) 2022-06-29 08:49:22 +01:00
Asim Aslam 1cd7cfaa6c go-micro.dev/v4 (#2305)
Run tests / Test repo (push) Has been cancelled
2021-10-12 12:55:53 +01:00
Kurisu_Amatist efd4ef0e62 fix(cache): only watch calling service in registry (#2273) 2021-09-19 10:40:21 +01:00
Asim Aslam d94936f6c9 v3 (#2104)
* v3

* revert plugins

* fixup some issues
2021-01-20 13:54:31 +00:00
Goober bf4ab679e1 Fix zk watchDir (#2102)
Signed-off-by: Goober <chenhao86899@gmail.com>
2021-01-05 17:32:17 +00:00
Hao b5431de2e8 Use singleflight to prevent registry cache breakdown (#2098) 2020-12-21 14:34:59 +00:00
Ben Toogood e17825474f Add context options to the runtime 2020-04-14 12:32:59 +01:00
Asim Aslam e5268dd0a6 move reg util to own package (#1523)
* move reg util to own package

* fix test

* fix broken static router
2020-04-10 17:41:10 +01:00
Vasiliy Tolstov 1fbc056dd4 minimize allocations (#1472)
* server: minimize allocations on re-register

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* server: stop old instance before Init()

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* client/grpc: fix allocations in protobuf marshal

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* codec/json: fix allocations in protobuf marshal

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* remove stop from init

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* codec/grpc: expose MaxMessageSize

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* codec: use buffer pool

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* metadata: minimize reallocations

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* util/wrapper: use metadata helper

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* registry/cache: move logs to debug level

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* server: move logs to debug level

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* server: cache service only when Advertise is ip addr

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* server: use metadata.Copy

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2020-04-08 10:50:19 +01:00
Vasiliy Tolstov 7b385bf163 minimize allocations in logger and tunnel code (#1323)
* logs alloc

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* fix allocs

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* fix allocs

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* tunnel allocs

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* try to fix tunnel

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* cache cipher for send

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* more logger

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* more logger

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* more logger

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* more logger

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* more logger

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* more logger

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* more logger

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2020-03-11 17:55:39 +00:00
Vasiliy Tolstov 117f56ebf7 prune util/log and user logger (#1237)
* prune util/log and user logger

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* plaintext logger

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>

* add newline

Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2020-02-23 13:45:20 +00:00
Vasiliy Tolstov f23638c036 fix import paths for v2 release
Signed-off-by: Vasiliy Tolstov <v.tolstov@unistack.org>
2020-01-30 14:44:40 +03:00
Asim Aslam 254045e9f3 Remove go routines for mdns watcher and cache registry (#919) 2019-11-06 15:49:40 +00:00
yandd f9c639af4e fix rcache ttl 2019-10-30 14:01:51 +08:00
Asim Aslam 44b794722e rcache becomes cache 2019-10-14 22:39:26 +01:00
Milos Gajdos fc89c9831e heck cache status error 2019-10-11 10:47:42 +01:00
Asim Aslam cfb846ee7e Fix race in cache 2019-10-04 16:40:21 +01:00
Asim Aslam c4b6d0f3a8 fix major deadlock in registry cache 2019-10-04 16:29:56 +01:00
Asim Aslam 5d77ce9e9b Rename rcache file to cache 2019-10-02 12:35:20 +01:00