Run Tests / Etcd Integration Tests (push) Has been cancelled
Provides a test harness for running micro services in isolation:
h := testing.NewHarness(t)
defer h.Stop()
h.Name("users").Register(new(UsersHandler))
h.Start()
var rsp Response
h.Call("UsersHandler.Create", &req, &rsp)
Features:
- Isolated registry, transport, and broker per harness
- Simple API: Name(), Register(), Start(), Stop()
- Call helpers: Call(), CallContext()
- Assertions: AssertServiceRunning(), AssertCallSucceeds(), AssertCallFails()
- Access to underlying Client(), Server(), Registry()
Note: Due to go-micro's global defaults, each harness tests one service.
For multi-service testing, use integration tests or mocks.
Also fixes README.md example showing conflicting port 8080.
Co-authored-by: Shelley <shelley@exe.dev>
micro run now starts an HTTP gateway alongside your services:
- Web dashboard at http://localhost:8080
- API proxy at /api/{service}/{method}
- Health checks at /health
- Service listing at /services
The experience is now:
$ micro new helloworld
$ cd helloworld
$ micro run
Open http://localhost:8080 to see and call your services.
New flags:
--address :3000 # Custom gateway port
--no-gateway # Disable gateway (services only)
Updated documentation to make this the central experience.
Co-authored-by: Shelley <shelley@exe.dev>
Adds a new health package providing:
- /health endpoint for overall health status
- /health/live for Kubernetes liveness probes
- /health/ready for Kubernetes readiness probes
- Built-in checks: PingCheck, TCPCheck, HTTPCheck, DNSCheck
- Critical vs non-critical checks
- Concurrent check execution
- Configurable timeouts
- Service info metadata
Integrates with micro run's health check waiting when services
specify a port in their micro.mu configuration.
Example usage:
health.Register("database", health.PingCheck(db.Ping))
health.Register("redis", health.TCPCheck("localhost:6379", time.Second))
health.RegisterHandlers(mux)
Co-authored-by: Shelley <shelley@exe.dev>
- Add micro.mu DSL and micro.json config file support
- Implement hot reload with file watching (--no-watch to disable)
- Start services in dependency order (topological sort)
- Environment management (--env flag, MICRO_ENV var)
- Health check waiting before starting dependents
- Graceful shutdown in reverse dependency order
Config file example (micro.mu):
service users
path ./users
port 8081
service posts
path ./posts
port 8082
depends users
env development
STORE_ADDRESS file://./data
Closes#2828
Co-authored-by: Shelley <shelley@exe.dev>
* fix: remove deprecated rand.Seed calls
Go 1.20+ automatically seeds the global random number generator.
These calls are no-ops and generate warnings with newer Go versions.
Removed from:
- selector/strategy.go
- registry/cache/cache.go
- broker/memory.go
- broker/http.go
- cmd/cmd.go
- transport/memory.go
Co-authored-by: Shelley <shelley@exe.dev>
* fix: handle previously ignored errors
- MySQL store: properly handle prepared statement errors in initDB()
- Consul registry: handle client creation errors in Client() method
These silent failures could cause hard-to-debug issues in production.
Co-authored-by: Shelley <shelley@exe.dev>
* feat(genai): improve provider interface with context and streaming
Breaking changes:
- Generate() and Stream() now require context.Context as first parameter
- Stream.Close() added for proper resource cleanup
Improvements:
- Proper context support for cancellation and timeouts
- Real SSE streaming for OpenAI and Gemini text generation
- Better error handling with wrapped errors and API error responses
- Thread-safe provider registry with sync.RWMutex
- New options: WithMaxTokens, WithTemperature, WithTimeout
- Stream has proper Close() method for cleanup
- Results can include Error field for per-chunk errors
Provider updates:
- OpenAI: true streaming with SSE parsing, proper HTTP client with timeout
- Gemini: true streaming with streamGenerateContent endpoint
- Default model updated to gpt-4o-mini (OpenAI) and gemini-2.0-flash (Gemini)
Co-authored-by: Shelley <shelley@exe.dev>
* feat(tls): make TLS secure by default, configurable via environment
BREAKING: TLS now verifies certificates by default. Set MICRO_TLS_INSECURE=true
to restore previous behavior (NOT recommended for production).
Changes:
- Add util/tls.Config(), SecureConfig(), InsecureConfig(), ConfigFromEnv() helpers
- Update all components to use ConfigFromEnv() instead of hardcoded InsecureSkipVerify
- Set MinVersion to TLS 1.2 for all TLS configs
Affected components:
- broker/http
- broker/rabbitmq
- registry/etcd
- registry/consul
- transport/grpc
This improves security posture while allowing opt-out for development environments.
Co-authored-by: Shelley <shelley@exe.dev>
* feat(tls): add TLS helpers with opt-in secure mode
NOT a breaking change - keeps InsecureSkipVerify=true as default for
local development compatibility.
New util/tls helpers:
- Config() - returns config based on MICRO_TLS_SECURE env var
- SecureConfig() - certificate verification enabled
- InsecureConfig() - certificate verification disabled (dev only)
For production security, use one of:
- Set MICRO_TLS_SECURE=true with proper CA-signed certs
- Use a service mesh (Istio, Linkerd) for automatic mTLS
- Configure TLSConfig directly with your certificates
Also: Changed CLI alias from 'g' to 'gen' for clarity
- micro generate handler -> micro gen handler
Co-authored-by: Shelley <shelley@exe.dev>
* refactor(cli): rename generate directory to gen for consistency
Directory name now matches the command alias:
cmd/micro/cli/gen/ -> micro gen handler
Co-authored-by: Shelley <shelley@exe.dev>
---------
Co-authored-by: Shelley <shelley@exe.dev>
This adds Server-Sent Events (SSE) support to the web package, enabling
real-time updates in web applications built with go-micro.
Features:
- SSEBroadcaster manages connected clients and broadcasts events
- Integrates with events.Stream for subscribing to topics
- Supports JSON events and raw HTML (for htmx/datastar)
- Keep-alive mechanism for long-lived connections
- Thread-safe client management
Usage:
broadcaster := web.NewSSEBroadcaster(
web.WithStream(stream),
web.WithTopics("posts", "comments"),
)
broadcaster.Start()
router.GET("/events", broadcaster.Handler())
This enables real-time features requested in #2788, including:
- Live updates without page refresh
- htmx/datastar integration
- Foundation for MCP (Model Context Protocol) support
Closes#2788
Co-authored-by: Shelley <shelley@exe.dev>
* Initial plan
* Enhance cache rate limiting to protect against rolling deployment scenarios
Per @asim's feedback, rate limiting now applies to ALL cache refresh attempts
(not just error cases) to prevent registry overload during rolling deployments.
Changes:
- Rate limit ALL refresh attempts using lastRefreshAttempt tracking
- Always return stale cache if available (even if expired) when rate limiting
- Only return ErrNotFound when no cache exists during rate limit period
- Rate limiting happens inside singleflight to avoid race conditions
- Update test to reflect new behavior (no retry when stale cache + rate limit)
- Enhanced documentation with rolling deployment scenario examples
This addresses the scenario where all upstream services expire their cache
simultaneously during a downstream rolling deployment, which would previously
cause a stampede to the registry under high QPS.
Co-authored-by: asim <17530+asim@users.noreply.github.com>
* Remove unused failedAttempts and consecutiveFailures fields
Per @asim's feedback, these fields are no longer needed since the new
rate limiting strategy uses lastRefreshAttempt to track ALL refresh
attempts, not just failed ones.
Removed:
- failedAttempts map[string]time.Time
- consecutiveFailures int
- All code that sets these fields
The rate limiting logic now only uses lastRefreshAttempt, making these
fields redundant. All tests continue to pass.
Co-authored-by: asim <17530+asim@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
* Initial plan
* Add adaptive throttling to prevent cache penetration without stale cache
Implement rate limiting mechanism to protect etcd when:
- Cache is empty (no stale data available)
- Registry is failing/overloaded
- Multiple sequential requests arrive
Changes:
- Add MinimumRetryInterval option (default 5s)
- Track last failed attempt time per service
- Throttle requests within retry interval when no cache exists
- Clear throttling on successful lookup
- Add comprehensive tests validating throttling behavior
Co-authored-by: asim <17530+asim@users.noreply.github.com>
* Fix WaitGroup race condition in events stream tests
The WaitGroup.Add() calls were happening after goroutines started,
causing a race where Done() could be called before Add(), resulting
in "negative WaitGroup counter" panics.
Fixes:
- Move wg.Add(1) before goroutine in TestConsumeTopic
- Move wg.Add(2) before first goroutine in TestConsumeGroup
Co-authored-by: asim <17530+asim@users.noreply.github.com>
* Update registry cache documentation with adaptive throttling feature
Document the new adaptive throttling feature that prevents cache
penetration when registry is failing and no stale cache exists.
Includes:
- Feature overview
- Usage examples with configuration options
- Explanation of throttling behavior
- Example scenario demonstrating protection
Co-authored-by: asim <17530+asim@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
* Initial plan
* Replace KeepAliveOnce with KeepAlive to reduce etcd auth requests
Co-authored-by: asim <17530+asim@users.noreply.github.com>
* Add tests to verify cache penetration protection via singleflight
Co-authored-by: asim <17530+asim@users.noreply.github.com>
* Add etcd integration tests to CI workflow
Co-authored-by: asim <17530+asim@users.noreply.github.com>
* Fix race conditions and improve code quality based on review feedback
Co-authored-by: asim <17530+asim@users.noreply.github.com>
* Add workflow permissions to fix security scan finding
Co-authored-by: asim <17530+asim@users.noreply.github.com>
* Add comprehensive documentation for performance improvements
Co-authored-by: asim <17530+asim@users.noreply.github.com>
* Fix memory store limit/offset bug causing events test failure
Co-authored-by: asim <17530+asim@users.noreply.github.com>
* Fix memory store to filter before limiting in prefix/suffix reads
The previous fix had a logic error where limit/offset were applied before
prefix/suffix filtering. This could cause incorrect results when the first
N items in the unfiltered list don't match the search criteria.
Now filters first to get all matching keys, then applies limit/offset to
the filtered results. This ensures ReadLimit(1) always returns 1 matching
record if available, regardless of map iteration order.
Co-authored-by: asim <17530+asim@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
* Initial plan
* docs: clarify gRPC server option ordering and service name usage
- Fix all examples to show Server option before Name option
- Add note explaining why option ordering matters
- Add new section on "Option Ordering Issue" in Common Errors
- Add new section on "Service Name vs Package Name" in Common Errors
- Update transport.md to show proper ordering with comment
Co-authored-by: asim <17530+asim@users.noreply.github.com>
* docs: refine comments based on code review feedback
- Clarify that Server ordering before Name is mandatory
- Remove confusing comment about Client ordering being for consistency
Co-authored-by: asim <17530+asim@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: asim <17530+asim@users.noreply.github.com>
Shelley