ingester

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 2, 2019 License: Apache-2.0 Imports: 49 Imported by: 1

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Bucket added in v0.4.0 

type Bucket struct {
	UserID string
	Bucket objstore.Bucket
}

Bucket is a wrapper around a objstore.Bucket that prepends writes with a userID

func (*Bucket) Close added in v0.4.0 

func (b *Bucket) Close() error

Close implements io.Closer

func (*Bucket) Delete added in v0.4.0 

func (b *Bucket) Delete(ctx context.Context, name string) error

Delete removes the object with the given name.

func (*Bucket) Exists added in v0.4.0 

func (b *Bucket) Exists(ctx context.Context, name string) (bool, error)

Exists checks if the given object exists in the bucket.

func (*Bucket) Get added in v0.4.0 

func (b *Bucket) Get(ctx context.Context, name string) (io.ReadCloser, error)

Get returns a reader for the given object name.

func (*Bucket) GetRange added in v0.4.0 

func (b *Bucket) GetRange(ctx context.Context, name string, off, length int64) (io.ReadCloser, error)

GetRange returns a new range reader for the given object name and range.

func (*Bucket) IsObjNotFoundErr added in v0.4.0 

func (b *Bucket) IsObjNotFoundErr(err error) bool

IsObjNotFoundErr returns true if error means that object is not found. Relevant to Get operations.

func (*Bucket) Iter added in v0.4.0 

func (b *Bucket) Iter(ctx context.Context, dir string, f func(string) error) error

Iter calls f for each entry in the given directory (not recursive.). The argument to f is the full object name including the prefix of the inspected directory.

func (*Bucket) Name added in v0.4.0 

func (b *Bucket) Name() string

Name returns the bucket name for the provider.

func (*Bucket) Upload added in v0.4.0 

func (b *Bucket) Upload(ctx context.Context, name string, r io.Reader) error

Upload the contents of the reader as an object into the bucket.

type ChunkStore 

type ChunkStore interface {
	Put(ctx context.Context, chunks []cortex_chunk.Chunk) error
}

ChunkStore is the interface we need to store chunks

type Config 

type Config struct {
	LifecyclerConfig ring.LifecyclerConfig `yaml:"lifecycler,omitempty"`

	// Config for transferring chunks. Zero or negative = no retries.
	MaxTransferRetries int `yaml:"max_transfer_retries,omitempty"`

	// Config for chunk flushing.
	FlushCheckPeriod  time.Duration
	RetainPeriod      time.Duration
	MaxChunkIdle      time.Duration
	MaxStaleChunkIdle time.Duration
	FlushOpTimeout    time.Duration
	MaxChunkAge       time.Duration
	ChunkAgeJitter    time.Duration
	ConcurrentFlushes int
	SpreadFlushes     bool

	RateUpdatePeriod time.Duration

	// Use tsdb block storage
	TSDBEnabled bool        `yaml:"-"`
	TSDBConfig  tsdb.Config `yaml:"-"`

	// Injected at runtime and read from the distributor config, required
	// to accurately apply global limits.
	ShardByAllLabels bool `yaml:"-"`
	// contains filtered or unexported fields
}

Config for an Ingester.

func (*Config) RegisterFlags 

func (cfg *Config) RegisterFlags(f *flag.FlagSet)

RegisterFlags adds the flags required to config this to the given FlagSet

type Ingester 

type Ingester struct {

	// Prometheus block storage
	TSDBState TSDBState
	// contains filtered or unexported fields
}

Ingester deals with "in flight" chunks. Based on Prometheus 1.x MemorySeriesStorage.

func New 

func New(cfg Config, clientConfig client.Config, limits *validation.Overrides, chunkStore ChunkStore, registerer prometheus.Registerer) (*Ingester, error)

New constructs a new Ingester.

func NewV2 added in v0.4.0 

func NewV2(cfg Config, clientConfig client.Config, limits *validation.Overrides, chunkStore ChunkStore, registerer prometheus.Registerer) (*Ingester, error)

NewV2 returns a new Ingester that uses prometheus block storage instead of chunk storage

func (*Ingester) AllUserStats 

func (i *Ingester) AllUserStats(ctx old_ctx.Context, req *client.UserStatsRequest) (*client.UsersStatsResponse, error)

AllUserStats returns ingestion statistics for all users known to this ingester.

func (*Ingester) Check 

func (i *Ingester) Check(ctx old_ctx.Context, req *grpc_health_v1.HealthCheckRequest) (*grpc_health_v1.HealthCheckResponse, error)

Check implements the grpc healthcheck

func (*Ingester) Flush 

func (i *Ingester) Flush()

Flush triggers a flush of all the chunks and closes the flush queues. Called from the Lifecycler as part of the ingester shutdown.

func (*Ingester) FlushHandler 

func (i *Ingester) FlushHandler(w http.ResponseWriter, r *http.Request)

FlushHandler triggers a flush of all in memory chunks. Mainly used for local testing.

func (*Ingester) LabelNames 

func (i *Ingester) LabelNames(ctx old_ctx.Context, req *client.LabelNamesRequest) (*client.LabelNamesResponse, error)

LabelNames return all the label names.

func (*Ingester) LabelValues 

func (i *Ingester) LabelValues(ctx old_ctx.Context, req *client.LabelValuesRequest) (*client.LabelValuesResponse, error)

LabelValues returns all label values that are associated with a given label name.

func (*Ingester) MetricsForLabelMatchers 

func (i *Ingester) MetricsForLabelMatchers(ctx old_ctx.Context, req *client.MetricsForLabelMatchersRequest) (*client.MetricsForLabelMatchersResponse, error)

MetricsForLabelMatchers returns all the metrics which match a set of matchers.

func (*Ingester) Push 

func (i *Ingester) Push(ctx old_ctx.Context, req *client.WriteRequest) (*client.WriteResponse, error)

Push implements client.IngesterServer

func (*Ingester) Query 

func (i *Ingester) Query(ctx old_ctx.Context, req *client.QueryRequest) (*client.QueryResponse, error)

Query implements service.IngesterServer

func (*Ingester) QueryStream 

func (i *Ingester) QueryStream(req *client.QueryRequest, stream client.Ingester_QueryStreamServer) error

QueryStream implements service.IngesterServer

func (*Ingester) ReadinessHandler 

func (i *Ingester) ReadinessHandler(w http.ResponseWriter, r *http.Request)

ReadinessHandler is used to indicate to k8s when the ingesters are ready for the addition removal of another ingester. Returns 204 when the ingester is ready, 500 otherwise.

func (*Ingester) Shutdown 

func (i *Ingester) Shutdown()

Shutdown beings the process to stop this ingester.

func (*Ingester) ShutdownHandler added in v0.4.0 

func (i *Ingester) ShutdownHandler(w http.ResponseWriter, r *http.Request)

ShutdownHandler triggers the following set of operations in order:

  • Change the state of ring to stop accepting writes.
  • Flush all the chunks.

func (*Ingester) StopIncomingRequests 

func (i *Ingester) StopIncomingRequests()

StopIncomingRequests is called during the shutdown process.

func (*Ingester) TransferChunks 

func (i *Ingester) TransferChunks(stream client.Ingester_TransferChunksServer) error

TransferChunks receives all the chunks from another ingester.

func (*Ingester) TransferOut 

func (i *Ingester) TransferOut(ctx context.Context) error

TransferOut finds an ingester in PENDING state and transfers our chunks to it. Called as part of the ingester shutdown process.

func (*Ingester) TransferTSDB added in v0.4.0 

func (i *Ingester) TransferTSDB(stream client.Ingester_TransferTSDBServer) error

TransferTSDB receives all the file chunks from another ingester, and writes them to tsdb directories

func (*Ingester) UserStats 

func (i *Ingester) UserStats(ctx old_ctx.Context, req *client.UserStatsRequest) (*client.UserStatsResponse, error)

UserStats returns ingestion statistics for the current user.

func (*Ingester) Watch 

func (i *Ingester) Watch(in *grpc_health_v1.HealthCheckRequest, stream grpc_health_v1.Health_WatchServer) error

Watch implements the grpc healthcheck.

type RingCount added in v0.4.0 

type RingCount interface {
	HealthyInstancesCount() int
}

RingCount is the interface exposed by a ring implementation which allows to count members

type SeriesLimiter added in v0.4.0 

type SeriesLimiter struct {
	// contains filtered or unexported fields
}

SeriesLimiter implements primitives to get the maximum number of series an ingester can handle for a specific tenant

func NewSeriesLimiter added in v0.4.0 

func NewSeriesLimiter(limits *validation.Overrides, ring RingCount, replicationFactor int, shardByAllLabels bool) *SeriesLimiter

NewSeriesLimiter makes a new in-memory series limiter

func (*SeriesLimiter) AssertMaxSeriesPerMetric added in v0.4.0 

func (l *SeriesLimiter) AssertMaxSeriesPerMetric(userID string, series int) error

AssertMaxSeriesPerMetric limit has not been reached compared to the current number of series in input and returns an error if so.

func (*SeriesLimiter) AssertMaxSeriesPerUser added in v0.4.0 

func (l *SeriesLimiter) AssertMaxSeriesPerUser(userID string, series int) error

AssertMaxSeriesPerUser limit has not been reached compared to the current number of series in input and returns an error if so.

func (*SeriesLimiter) MaxSeriesPerQuery added in v0.4.0 

func (l *SeriesLimiter) MaxSeriesPerQuery(userID string) int

MaxSeriesPerQuery returns the maximum number of series a query is allowed to hit.

type TSDBState added in v0.4.0 

type TSDBState struct {
	// contains filtered or unexported fields
}

TSDBState holds data structures used by the TSDB storage engine

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.