jonathanjordan21 commited on
Commit
7a836ed
·
verified ·
1 Parent(s): f87bfee

Update user.yaml

Browse files
Files changed (1) hide show
  1. user.yaml +1091 -0
user.yaml CHANGED
@@ -0,0 +1,1091 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Licensed to the LF AI & Data foundation under one
2
+ # or more contributor license agreements. See the NOTICE file
3
+ # distributed with this work for additional information
4
+ # regarding copyright ownership. The ASF licenses this file
5
+ # to you under the Apache License, Version 2.0 (the
6
+ # "License"); you may not use this file except in compliance
7
+ # with the License. You may obtain a copy of the License at
8
+ #
9
+ # http://www.apache.org/licenses/LICENSE-2.0
10
+ #
11
+ # Unless required by applicable law or agreed to in writing, software
12
+ # distributed under the License is distributed on an "AS IS" BASIS,
13
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14
+ # See the License for the specific language governing permissions and
15
+ # limitations under the License.
16
+
17
+ # Related configuration of etcd, used to store Milvus metadata & service discovery.
18
+ etcd:
19
+ # Endpoints used to access etcd service. You can change this parameter as the endpoints of your own etcd cluster.
20
+ # Environment variable: ETCD_ENDPOINTS
21
+ # etcd preferentially acquires valid address from environment variable ETCD_ENDPOINTS when Milvus is started.
22
+ endpoints: localhost:2379
23
+ # Root prefix of the key to where Milvus stores data in etcd.
24
+ # It is recommended to change this parameter before starting Milvus for the first time.
25
+ # To share an etcd instance among multiple Milvus instances, consider changing this to a different value for each Milvus instance before you start them.
26
+ # Set an easy-to-identify root path for Milvus if etcd service already exists.
27
+ # Changing this for an already running Milvus instance may result in failures to read legacy data.
28
+ rootPath: by-dev
29
+ # Sub-prefix of the key to where Milvus stores metadata-related information in etcd.
30
+ # Caution: Changing this parameter after using Milvus for a period of time will affect your access to old data.
31
+ # It is recommended to change this parameter before starting Milvus for the first time.
32
+ metaSubPath: meta
33
+ # Sub-prefix of the key to where Milvus stores timestamps in etcd.
34
+ # Caution: Changing this parameter after using Milvus for a period of time will affect your access to old data.
35
+ # It is recommended not to change this parameter if there is no specific reason.
36
+ kvSubPath: kv
37
+ log:
38
+ level: info # Only supports debug, info, warn, error, panic, or fatal. Default 'info'.
39
+ # path is one of:
40
+ # - "default" as os.Stderr,
41
+ # - "stderr" as os.Stderr,
42
+ # - "stdout" as os.Stdout,
43
+ # - file path to append server logs to.
44
+ # please adjust in embedded Milvus: /tmp/milvus/logs/etcd.log
45
+ path: stdout
46
+ ssl:
47
+ enabled: false # Whether to support ETCD secure connection mode
48
+ tlsCert: /path/to/etcd-client.pem # path to your cert file
49
+ tlsKey: /path/to/etcd-client-key.pem # path to your key file
50
+ tlsCACert: /path/to/ca.pem # path to your CACert file
51
+ # TLS min version
52
+ # Optional values: 1.0, 1.1, 1.2, 1.3。
53
+ # We recommend using version 1.2 and above.
54
+ tlsMinVersion: 1.3
55
+ requestTimeout: 10000 # Etcd operation timeout in milliseconds
56
+ use:
57
+ embed: false # Whether to enable embedded Etcd (an in-process EtcdServer).
58
+ data:
59
+ dir: default.etcd # Embedded Etcd only. please adjust in embedded Milvus: /tmp/milvus/etcdData/
60
+ auth:
61
+ enabled: false # Whether to enable authentication
62
+ userName: # username for etcd authentication
63
+ password: # password for etcd authentication
64
+
65
+ metastore:
66
+ type: etcd # Default value: etcd, Valid values: [etcd, tikv]
67
+ snapshot:
68
+ ttl: 86400 # snapshot ttl in seconds
69
+ reserveTime: 3600 # snapshot reserve time in seconds
70
+
71
+ # Related configuration of tikv, used to store Milvus metadata.
72
+ # Notice that when TiKV is enabled for metastore, you still need to have etcd for service discovery.
73
+ # TiKV is a good option when the metadata size requires better horizontal scalability.
74
+ tikv:
75
+ endpoints: 127.0.0.1:2389 # Note that the default pd port of tikv is 2379, which conflicts with etcd.
76
+ rootPath: by-dev # The root path where data is stored in tikv
77
+ metaSubPath: meta # metaRootPath = rootPath + '/' + metaSubPath
78
+ kvSubPath: kv # kvRootPath = rootPath + '/' + kvSubPath
79
+ requestTimeout: 10000 # ms, tikv request timeout
80
+ snapshotScanSize: 256 # batch size of tikv snapshot scan
81
+ ssl:
82
+ enabled: false # Whether to support TiKV secure connection mode
83
+ tlsCert: # path to your cert file
84
+ tlsKey: # path to your key file
85
+ tlsCACert: # path to your CACert file
86
+
87
+ localStorage:
88
+ # Local path to where vector data are stored during a search or a query to avoid repetitve access to MinIO or S3 service.
89
+ # Caution: Changing this parameter after using Milvus for a period of time will affect your access to old data.
90
+ # It is recommended to change this parameter before starting Milvus for the first time.
91
+ path: /var/lib/milvus/data/
92
+
93
+ # Related configuration of MinIO/S3/GCS or any other service supports S3 API, which is responsible for data persistence for Milvus.
94
+ # We refer to the storage service as MinIO/S3 in the following description for simplicity.
95
+ minio:
96
+ # IP address of MinIO or S3 service.
97
+ # Environment variable: MINIO_ADDRESS
98
+ # minio.address and minio.port together generate the valid access to MinIO or S3 service.
99
+ # MinIO preferentially acquires the valid IP address from the environment variable MINIO_ADDRESS when Milvus is started.
100
+ # Default value applies when MinIO or S3 is running on the same network with Milvus.
101
+ address: localhost
102
+ port: 9000 # Port of MinIO or S3 service.
103
+ # Access key ID that MinIO or S3 issues to user for authorized access.
104
+ # Environment variable: MINIO_ACCESS_KEY_ID or minio.accessKeyID
105
+ # minio.accessKeyID and minio.secretAccessKey together are used for identity authentication to access the MinIO or S3 service.
106
+ # This configuration must be set identical to the environment variable MINIO_ACCESS_KEY_ID, which is necessary for starting MinIO or S3.
107
+ # The default value applies to MinIO or S3 service that started with the default docker-compose.yml file.
108
+ accessKeyID: minioadmin
109
+ # Secret key used to encrypt the signature string and verify the signature string on server. It must be kept strictly confidential and accessible only to the MinIO or S3 server and users.
110
+ # Environment variable: MINIO_SECRET_ACCESS_KEY or minio.secretAccessKey
111
+ # minio.accessKeyID and minio.secretAccessKey together are used for identity authentication to access the MinIO or S3 service.
112
+ # This configuration must be set identical to the environment variable MINIO_SECRET_ACCESS_KEY, which is necessary for starting MinIO or S3.
113
+ # The default value applies to MinIO or S3 service that started with the default docker-compose.yml file.
114
+ secretAccessKey: minioadmin
115
+ useSSL: false # Switch value to control if to access the MinIO or S3 service through SSL.
116
+ ssl:
117
+ tlsCACert: /path/to/public.crt # path to your CACert file
118
+ # Name of the bucket where Milvus stores data in MinIO or S3.
119
+ # Milvus 2.0.0 does not support storing data in multiple buckets.
120
+ # Bucket with this name will be created if it does not exist. If the bucket already exists and is accessible, it will be used directly. Otherwise, there will be an error.
121
+ # To share an MinIO instance among multiple Milvus instances, consider changing this to a different value for each Milvus instance before you start them. For details, see Operation FAQs.
122
+ # The data will be stored in the local Docker if Docker is used to start the MinIO service locally. Ensure that there is sufficient storage space.
123
+ # A bucket name is globally unique in one MinIO or S3 instance.
124
+ bucketName: a-bucket
125
+ # Root prefix of the key to where Milvus stores data in MinIO or S3.
126
+ # It is recommended to change this parameter before starting Milvus for the first time.
127
+ # To share an MinIO instance among multiple Milvus instances, consider changing this to a different value for each Milvus instance before you start them. For details, see Operation FAQs.
128
+ # Set an easy-to-identify root key prefix for Milvus if etcd service already exists.
129
+ # Changing this for an already running Milvus instance may result in failures to read legacy data.
130
+ rootPath: files
131
+ # Whether to useIAM role to access S3/GCS instead of access/secret keys
132
+ # For more information, refer to
133
+ # aws: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use.html
134
+ # gcp: https://cloud.google.com/storage/docs/access-control/iam
135
+ # aliyun (ack): https://www.alibabacloud.com/help/en/container-service-for-kubernetes/latest/use-rrsa-to-enforce-access-control
136
+ # aliyun (ecs): https://www.alibabacloud.com/help/en/elastic-compute-service/latest/attach-an-instance-ram-role
137
+ useIAM: false
138
+ # Cloud Provider of S3. Supports: "aws", "gcp", "aliyun".
139
+ # Cloud Provider of Google Cloud Storage. Supports: "gcpnative".
140
+ # You can use "aws" for other cloud provider supports S3 API with signature v4, e.g.: minio
141
+ # You can use "gcp" for other cloud provider supports S3 API with signature v2
142
+ # You can use "aliyun" for other cloud provider uses virtual host style bucket
143
+ # You can use "gcpnative" for the Google Cloud Platform provider. Uses service account credentials
144
+ # for authentication.
145
+ # When useIAM enabled, only "aws", "gcp", "aliyun" is supported for now
146
+ cloudProvider: aws
147
+ # The JSON content contains the gcs service account credentials.
148
+ # Used only for the "gcpnative" cloud provider.
149
+ gcpCredentialJSON:
150
+ # Custom endpoint for fetch IAM role credentials. when useIAM is true & cloudProvider is "aws".
151
+ # Leave it empty if you want to use AWS default endpoint
152
+ iamEndpoint:
153
+ logLevel: fatal # Log level for aws sdk log. Supported level: off, fatal, error, warn, info, debug, trace
154
+ region: # Specify minio storage system location region
155
+ useVirtualHost: false # Whether use virtual host mode for bucket
156
+ requestTimeoutMs: 10000 # minio timeout for request time in milliseconds
157
+ # The maximum number of objects requested per batch in minio ListObjects rpc,
158
+ # 0 means using oss client by default, decrease these configration if ListObjects timeout
159
+ listObjectsMaxKeys: 0
160
+
161
+ # Milvus supports four MQ: rocksmq(based on RockDB), natsmq(embedded nats-server), Pulsar and Kafka.
162
+ # You can change your mq by setting mq.type field.
163
+ # If you don't set mq.type field as default, there is a note about enabling priority if we config multiple mq in this file.
164
+ # 1. standalone(local) mode: rocksmq(default) > natsmq > Pulsar > Kafka
165
+ # 2. cluster mode: Pulsar(default) > Kafka (rocksmq and natsmq is unsupported in cluster mode)
166
+ mq:
167
+ # Default value: "default"
168
+ # Valid values: [default, pulsar, kafka, rocksmq, natsmq]
169
+ type: default
170
+ enablePursuitMode: true # Default value: "true"
171
+ pursuitLag: 10 # time tick lag threshold to enter pursuit mode, in seconds
172
+ pursuitBufferSize: 8388608 # pursuit mode buffer size in bytes
173
+ pursuitBufferTime: 60 # pursuit mode buffer time in seconds
174
+ mqBufSize: 16 # MQ client consumer buffer length
175
+ dispatcher:
176
+ mergeCheckInterval: 1 # the interval time(in seconds) for dispatcher to check whether to merge
177
+ targetBufSize: 16 # the lenth of channel buffer for targe
178
+ maxTolerantLag: 3 # Default value: "3", the timeout(in seconds) that target sends msgPack
179
+
180
+ # Related configuration of pulsar, used to manage Milvus logs of recent mutation operations, output streaming log, and provide log publish-subscribe services.
181
+ pulsar:
182
+ # IP address of Pulsar service.
183
+ # Environment variable: PULSAR_ADDRESS
184
+ # pulsar.address and pulsar.port together generate the valid access to Pulsar.
185
+ # Pulsar preferentially acquires the valid IP address from the environment variable PULSAR_ADDRESS when Milvus is started.
186
+ # Default value applies when Pulsar is running on the same network with Milvus.
187
+ address: localhost
188
+ port: 6650 # Port of Pulsar service.
189
+ webport: 80 # Web port of of Pulsar service. If you connect direcly without proxy, should use 8080.
190
+ # The maximum size of each message in Pulsar. Unit: Byte.
191
+ # By default, Pulsar can transmit at most 2MB of data in a single message. When the size of inserted data is greater than this value, proxy fragments the data into multiple messages to ensure that they can be transmitted correctly.
192
+ # If the corresponding parameter in Pulsar remains unchanged, increasing this configuration will cause Milvus to fail, and reducing it produces no advantage.
193
+ maxMessageSize: 2097152
194
+ # Pulsar can be provisioned for specific tenants with appropriate capacity allocated to the tenant.
195
+ # To share a Pulsar instance among multiple Milvus instances, you can change this to an Pulsar tenant rather than the default one for each Milvus instance before you start them. However, if you do not want Pulsar multi-tenancy, you are advised to change msgChannel.chanNamePrefix.cluster to the different value.
196
+ tenant: public
197
+ namespace: default # A Pulsar namespace is the administrative unit nomenclature within a tenant.
198
+ requestTimeout: 60 # pulsar client global request timeout in seconds
199
+ enableClientMetrics: false # Whether to register pulsar client metrics into milvus metrics path.
200
+
201
+ # If you want to enable kafka, needs to comment the pulsar configs
202
+ # kafka:
203
+ # brokerList: localhost:9092
204
+ # saslUsername:
205
+ # saslPassword:
206
+ # saslMechanisms:
207
+ # securityProtocol:
208
+ # ssl:
209
+ # enabled: false # whether to enable ssl mode
210
+ # tlsCert: # path to client's public key (PEM) used for authentication
211
+ # tlsKey: # path to client's private key (PEM) used for authentication
212
+ # tlsCaCert: # file or directory path to CA certificate(s) for verifying the broker's key
213
+ # tlsKeyPassword: # private key passphrase for use with ssl.key.location and set_ssl_cert(), if any
214
+ # readTimeout: 10
215
+
216
+ rocksmq:
217
+ # Prefix of the key to where Milvus stores data in RocksMQ.
218
+ # Caution: Changing this parameter after using Milvus for a period of time will affect your access to old data.
219
+ # It is recommended to change this parameter before starting Milvus for the first time.
220
+ # Set an easy-to-identify root key prefix for Milvus if etcd service already exists.
221
+ path: /var/lib/milvus/rdb_data
222
+ lrucacheratio: 0.06 # rocksdb cache memory ratio
223
+ rocksmqPageSize: 67108864 # The maximum size of messages in each page in RocksMQ. Messages in RocksMQ are checked and cleared (when expired) in batch based on this parameters. Unit: Byte.
224
+ retentionTimeInMinutes: 4320 # The maximum retention time of acked messages in RocksMQ. Acked messages in RocksMQ are retained for the specified period of time and then cleared. Unit: Minute.
225
+ retentionSizeInMB: 8192 # The maximum retention size of acked messages of each topic in RocksMQ. Acked messages in each topic are cleared if their size exceed this parameter. Unit: MB.
226
+ compactionInterval: 86400 # Time interval to trigger rocksdb compaction to remove deleted data. Unit: Second
227
+ compressionTypes: 0,0,7,7,7 # compaction compression type, only support use 0,7. 0 means not compress, 7 will use zstd. Length of types means num of rocksdb level.
228
+
229
+ # natsmq configuration.
230
+ # more detail: https://docs.nats.io/running-a-nats-service/configuration
231
+ natsmq:
232
+ server:
233
+ port: 4222 # Listening port of the NATS server.
234
+ storeDir: /var/lib/milvus/nats # Directory to use for JetStream storage of nats
235
+ maxFileStore: 17179869184 # Maximum size of the 'file' storage
236
+ maxPayload: 8388608 # Maximum number of bytes in a message payload
237
+ maxPending: 67108864 # Maximum number of bytes buffered for a connection Applies to client connections
238
+ initializeTimeout: 4000 # waiting for initialization of natsmq finished
239
+ monitor:
240
+ trace: false # If true enable protocol trace log messages
241
+ debug: false # If true enable debug log messages
242
+ logTime: true # If set to false, log without timestamps.
243
+ logFile: /tmp/milvus/logs/nats.log # Log file path relative to .. of milvus binary if use relative path
244
+ logSizeLimit: 536870912 # Size in bytes after the log file rolls over to a new one
245
+ retention:
246
+ maxAge: 4320 # Maximum age of any message in the P-channel
247
+ maxBytes: # How many bytes the single P-channel may contain. Removing oldest messages if the P-channel exceeds this size
248
+ maxMsgs: # How many message the single P-channel may contain. Removing oldest messages if the P-channel exceeds this limit
249
+
250
+ # Related configuration of rootCoord, used to handle data definition language (DDL) and data control language (DCL) requests
251
+ rootCoord:
252
+ dmlChannelNum: 16 # The number of DML-Channels to create at the root coord startup.
253
+ # The maximum number of partitions in each collection.
254
+ # New partitions cannot be created if this parameter is set as 0 or 1.
255
+ # Range: [0, INT64MAX]
256
+ maxPartitionNum: 1024
257
+ # The minimum row count of a segment required for creating index.
258
+ # Segments with smaller size than this parameter will not be indexed, and will be searched with brute force.
259
+ minSegmentSizeToEnableIndex: 1024
260
+ enableActiveStandby: false
261
+ maxDatabaseNum: 64 # Maximum number of database
262
+ maxGeneralCapacity: 65536 # upper limit for the sum of of product of partitionNumber and shardNumber
263
+ gracefulStopTimeout: 5 # seconds. force stop node without graceful stop
264
+ ip: # TCP/IP address of rootCoord. If not specified, use the first unicastable address
265
+ port: 53100 # TCP port of rootCoord
266
+ grpc:
267
+ serverMaxSendSize: 536870912 # The maximum size of each RPC request that the rootCoord can send, unit: byte
268
+ serverMaxRecvSize: 268435456 # The maximum size of each RPC request that the rootCoord can receive, unit: byte
269
+ clientMaxSendSize: 268435456 # The maximum size of each RPC request that the clients on rootCoord can send, unit: byte
270
+ clientMaxRecvSize: 536870912 # The maximum size of each RPC request that the clients on rootCoord can receive, unit: byte
271
+
272
+ # Related configuration of proxy, used to validate client requests and reduce the returned results.
273
+ proxy:
274
+ timeTickInterval: 200 # The interval at which proxy synchronizes the time tick, unit: ms.
275
+ healthCheckTimeout: 3000 # ms, the interval that to do component healthy check
276
+ msgStream:
277
+ timeTick:
278
+ bufSize: 512 # The maximum number of messages can be buffered in the timeTick message stream of the proxy when producing messages.
279
+ maxNameLength: 255 # The maximum length of the name or alias that can be created in Milvus, including the collection name, collection alias, partition name, and field name.
280
+ maxFieldNum: 64 # The maximum number of field can be created when creating in a collection. It is strongly DISCOURAGED to set maxFieldNum >= 64.
281
+ maxVectorFieldNum: 4 # The maximum number of vector fields that can be specified in a collection. Value range: [1, 10].
282
+ maxShardNum: 16 # The maximum number of shards can be created when creating in a collection.
283
+ maxDimension: 32768 # The maximum number of dimensions of a vector can have when creating in a collection.
284
+ # Whether to produce gin logs.\n
285
+ # please adjust in embedded Milvus: false
286
+ ginLogging: true
287
+ ginLogSkipPaths: / # skip url path for gin log
288
+ maxTaskNum: 1024 # The maximum number of tasks in the task queue of the proxy.
289
+ mustUsePartitionKey: false # switch for whether proxy must use partition key for the collection
290
+ accessLog:
291
+ enable: false # Whether to enable the access log feature.
292
+ minioEnable: false # Whether to upload local access log files to MinIO. This parameter can be specified when proxy.accessLog.filename is not empty.
293
+ localPath: /tmp/milvus_access # The local folder path where the access log file is stored. This parameter can be specified when proxy.accessLog.filename is not empty.
294
+ filename: # The name of the access log file. If you leave this parameter empty, access logs will be printed to stdout.
295
+ maxSize: 64 # The maximum size allowed for a single access log file. If the log file size reaches this limit, a rotation process will be triggered. This process seals the current access log file, creates a new log file, and clears the contents of the original log file. Unit: MB.
296
+ rotatedTime: 0 # The maximum time interval allowed for rotating a single access log file. Upon reaching the specified time interval, a rotation process is triggered, resulting in the creation of a new access log file and sealing of the previous one. Unit: seconds
297
+ remotePath: access_log/ # The path of the object storage for uploading access log files.
298
+ remoteMaxTime: 0 # The time interval allowed for uploading access log files. If the upload time of a log file exceeds this interval, the file will be deleted. Setting the value to 0 disables this feature.
299
+ formatters:
300
+ base:
301
+ format: "[$time_now] [ACCESS] <$user_name: $user_addr> $method_name [status: $method_status] [code: $error_code] [sdk: $sdk_version] [msg: $error_msg] [traceID: $trace_id] [timeCost: $time_cost]"
302
+ query:
303
+ format: "[$time_now] [ACCESS] <$user_name: $user_addr> $method_name [status: $method_status] [code: $error_code] [sdk: $sdk_version] [msg: $error_msg] [traceID: $trace_id] [timeCost: $time_cost] [database: $database_name] [collection: $collection_name] [partitions: $partition_name] [expr: $method_expr]"
304
+ methods: "Query,Search,Delete"
305
+ cacheSize: 0 # Size of log of write cache, in byte. (Close write cache if size was 0)
306
+ cacheFlushInterval: 3 # time interval of auto flush write cache, in seconds. (Close auto flush if interval was 0)
307
+ connectionCheckIntervalSeconds: 120 # the interval time(in seconds) for connection manager to scan inactive client info
308
+ connectionClientInfoTTLSeconds: 86400 # inactive client info TTL duration, in seconds
309
+ maxConnectionNum: 10000 # the max client info numbers that proxy should manage, avoid too many client infos
310
+ gracefulStopTimeout: 30 # seconds. force stop node without graceful stop
311
+ slowQuerySpanInSeconds: 5 # query whose executed time exceeds the `slowQuerySpanInSeconds` can be considered slow, in seconds.
312
+ queryNodePooling:
313
+ size: 10 # the size for shardleader(querynode) client pool
314
+ http:
315
+ enabled: true # Whether to enable the http server
316
+ debug_mode: false # Whether to enable http server debug mode
317
+ port: # high-level restful api
318
+ acceptTypeAllowInt64: true # high-level restful api, whether http client can deal with int64
319
+ enablePprof: true # Whether to enable pprof middleware on the metrics port
320
+ ip: # TCP/IP address of proxy. If not specified, use the first unicastable address
321
+ port: 19530 # TCP port of proxy
322
+ internalPort: 19529
323
+ grpc:
324
+ serverMaxSendSize: 268435456 # The maximum size of each RPC request that the proxy can send, unit: byte
325
+ serverMaxRecvSize: 67108864 # The maximum size of each RPC request that the proxy can receive, unit: byte
326
+ clientMaxSendSize: 268435456 # The maximum size of each RPC request that the clients on proxy can send, unit: byte
327
+ clientMaxRecvSize: 67108864 # The maximum size of each RPC request that the clients on proxy can receive, unit: byte
328
+
329
+ # Related configuration of queryCoord, used to manage topology and load balancing for the query nodes, and handoff from growing segments to sealed segments.
330
+ queryCoord:
331
+ taskMergeCap: 1
332
+ taskExecutionCap: 256
333
+ # Switch value to control if to automatically replace a growing segment with the corresponding indexed sealed segment when the growing segment reaches the sealing threshold.
334
+ # If this parameter is set false, Milvus simply searches the growing segments with brute force.
335
+ autoHandoff: true
336
+ autoBalance: true # Switch value to control if to automatically balance the memory usage among query nodes by distributing segment loading and releasing operations evenly.
337
+ autoBalanceChannel: true # Enable auto balance channel
338
+ balancer: ScoreBasedBalancer # auto balancer used for segments on queryNodes
339
+ globalRowCountFactor: 0.1 # the weight used when balancing segments among queryNodes
340
+ scoreUnbalanceTolerationFactor: 0.05 # the least value for unbalanced extent between from and to nodes when doing balance
341
+ reverseUnBalanceTolerationFactor: 1.3 # the largest value for unbalanced extent between from and to nodes after doing balance
342
+ overloadedMemoryThresholdPercentage: 90 # The threshold of memory usage (in percentage) in a query node to trigger the sealed segment balancing.
343
+ balanceIntervalSeconds: 60 # The interval at which query coord balances the memory usage among query nodes.
344
+ memoryUsageMaxDifferencePercentage: 30 # The threshold of memory usage difference (in percentage) between any two query nodes to trigger the sealed segment balancing.
345
+ rowCountFactor: 0.4 # the row count weight used when balancing segments among queryNodes
346
+ segmentCountFactor: 0.4 # the segment count weight used when balancing segments among queryNodes
347
+ globalSegmentCountFactor: 0.1 # the segment count weight used when balancing segments among queryNodes
348
+ segmentCountMaxSteps: 50 # segment count based plan generator max steps
349
+ rowCountMaxSteps: 50 # segment count based plan generator max steps
350
+ randomMaxSteps: 10 # segment count based plan generator max steps
351
+ growingRowCountWeight: 4 # the memory weight of growing segment row count
352
+ delegatorMemoryOverloadFactor: 0.1 # the factor of delegator overloaded memory
353
+ balanceCostThreshold: 0.001 # the threshold of balance cost, if the difference of cluster's cost after executing the balance plan is less than this value, the plan will not be executed
354
+ checkSegmentInterval: 1000
355
+ checkChannelInterval: 1000
356
+ checkBalanceInterval: 10000
357
+ checkIndexInterval: 10000
358
+ channelTaskTimeout: 60000 # 1 minute
359
+ segmentTaskTimeout: 120000 # 2 minute
360
+ distPullInterval: 500
361
+ heartbeatAvailableInterval: 10000 # 10s, Only QueryNodes which fetched heartbeats within the duration are available
362
+ loadTimeoutSeconds: 600
363
+ distRequestTimeout: 5000 # the request timeout for querycoord fetching data distribution from querynodes, in milliseconds
364
+ heatbeatWarningLag: 5000 # the lag value for querycoord report warning when last heatbeat is too old, in milliseconds
365
+ checkHandoffInterval: 5000
366
+ enableActiveStandby: false
367
+ checkInterval: 1000
368
+ checkHealthInterval: 3000 # 3s, the interval when query coord try to check health of query node
369
+ checkHealthRPCTimeout: 2000 # 100ms, the timeout of check health rpc to query node
370
+ brokerTimeout: 5000 # 5000ms, querycoord broker rpc timeout
371
+ collectionRecoverTimes: 3 # if collection recover times reach the limit during loading state, release it
372
+ observerTaskParallel: 16 # the parallel observer dispatcher task number
373
+ checkAutoBalanceConfigInterval: 10 # the interval of check auto balance config
374
+ checkNodeSessionInterval: 60 # the interval(in seconds) of check querynode cluster session
375
+ gracefulStopTimeout: 5 # seconds. force stop node without graceful stop
376
+ enableStoppingBalance: true # whether enable stopping balance
377
+ channelExclusiveNodeFactor: 4 # the least node number for enable channel's exclusive mode
378
+ collectionObserverInterval: 200 # the interval of collection observer
379
+ checkExecutedFlagInterval: 100 # the interval of check executed flag to force to pull dist
380
+ updateCollectionLoadStatusInterval: 5 # 5m, max interval of updating collection loaded status for check health
381
+ cleanExcludeSegmentInterval: 60 # the time duration of clean pipeline exclude segment which used for filter invalid data, in seconds
382
+ ip: # TCP/IP address of queryCoord. If not specified, use the first unicastable address
383
+ port: 19531 # TCP port of queryCoord
384
+ grpc:
385
+ serverMaxSendSize: 536870912 # The maximum size of each RPC request that the queryCoord can send, unit: byte
386
+ serverMaxRecvSize: 268435456 # The maximum size of each RPC request that the queryCoord can receive, unit: byte
387
+ clientMaxSendSize: 268435456 # The maximum size of each RPC request that the clients on queryCoord can send, unit: byte
388
+ clientMaxRecvSize: 536870912 # The maximum size of each RPC request that the clients on queryCoord can receive, unit: byte
389
+
390
+ # Related configuration of queryNode, used to run hybrid search between vector and scalar data.
391
+ queryNode:
392
+ stats:
393
+ publishInterval: 1000 # The interval that query node publishes the node statistics information, including segment status, cpu usage, memory usage, health status, etc. Unit: ms.
394
+ segcore:
395
+ knowhereThreadPoolNumRatio: 4 # The number of threads in knowhere's thread pool. If disk is enabled, the pool size will multiply with knowhereThreadPoolNumRatio([1, 32]).
396
+ chunkRows: 128 # Row count by which Segcore divides a segment into chunks.
397
+ interimIndex:
398
+ # Whether to create a temporary index for growing segments and sealed segments not yet indexed, improving search performance.
399
+ # Milvus will eventually seals and indexes all segments, but enabling this optimizes search performance for immediate queries following data insertion.
400
+ # This defaults to true, indicating that Milvus creates temporary index for growing segments and the sealed segments that are not indexed upon searches.
401
+ enableIndex: true
402
+ nlist: 128 # temp index nlist, recommend to set sqrt(chunkRows), must smaller than chunkRows/8
403
+ nprobe: 16 # nprobe to search small index, based on your accuracy requirement, must smaller than nlist
404
+ memExpansionRate: 1.15 # extra memory needed by building interim index
405
+ buildParallelRate: 0.5 # the ratio of building interim index parallel matched with cpu num
406
+ multipleChunkedEnable: true # Enable multiple chunked search
407
+ knowhereScoreConsistency: false # Enable knowhere strong consistency score computation logic
408
+ loadMemoryUsageFactor: 1 # The multiply factor of calculating the memory usage while loading segments
409
+ enableDisk: false # enable querynode load disk index, and search on disk index
410
+ maxDiskUsagePercentage: 95
411
+ cache:
412
+ memoryLimit: 2147483648 # 2 GB, 2 * 1024 *1024 *1024
413
+ readAheadPolicy: willneed # The read ahead policy of chunk cache, options: `normal, random, sequential, willneed, dontneed`
414
+ # options: async, sync, disable.
415
+ # Specifies the necessity for warming up the chunk cache.
416
+ # 1. If set to "sync" or "async" the original vector data will be synchronously/asynchronously loaded into the
417
+ # chunk cache during the load process. This approach has the potential to substantially reduce query/search latency
418
+ # for a specific duration post-load, albeit accompanied by a concurrent increase in disk usage;
419
+ # 2. If set to "disable" original vector data will only be loaded into the chunk cache during search/query.
420
+ warmup: disable
421
+ mmap:
422
+ vectorField: false # Enable mmap for loading vector data
423
+ vectorIndex: false # Enable mmap for loading vector index
424
+ scalarField: false # Enable mmap for loading scalar data
425
+ scalarIndex: false # Enable mmap for loading scalar index
426
+ chunkCache: true # Enable mmap for chunk cache (raw vector retrieving).
427
+ # Enable memory mapping (mmap) to optimize the handling of growing raw data.
428
+ # By activating this feature, the memory overhead associated with newly added or modified data will be significantly minimized.
429
+ # However, this optimization may come at the cost of a slight decrease in query latency for the affected data segments.
430
+ growingMmapEnabled: false
431
+ fixedFileSizeForMmapAlloc: 1 # tmp file size for mmap chunk manager
432
+ maxDiskUsagePercentageForMmapAlloc: 50 # disk percentage used in mmap chunk manager
433
+ lazyload:
434
+ enabled: false # Enable lazyload for loading data
435
+ waitTimeout: 30000 # max wait timeout duration in milliseconds before start to do lazyload search and retrieve
436
+ requestResourceTimeout: 5000 # max timeout in milliseconds for waiting request resource for lazy load, 5s by default
437
+ requestResourceRetryInterval: 2000 # retry interval in milliseconds for waiting request resource for lazy load, 2s by default
438
+ maxRetryTimes: 1 # max retry times for lazy load, 1 by default
439
+ maxEvictPerRetry: 1 # max evict count for lazy load, 1 by default
440
+ indexOffsetCacheEnabled: false # enable index offset cache for some scalar indexes, now is just for bitmap index, enable this param can improve performance for retrieving raw data from index
441
+ grouping:
442
+ enabled: true
443
+ maxNQ: 1000
444
+ topKMergeRatio: 20
445
+ scheduler:
446
+ receiveChanSize: 10240
447
+ unsolvedQueueSize: 10240
448
+ # maxReadConcurrentRatio is the concurrency ratio of read task (search task and query task).
449
+ # Max read concurrency would be the value of hardware.GetCPUNum * maxReadConcurrentRatio.
450
+ # It defaults to 2.0, which means max read concurrency would be the value of hardware.GetCPUNum * 2.
451
+ # Max read concurrency must greater than or equal to 1, and less than or equal to hardware.GetCPUNum * 100.
452
+ # (0, 100]
453
+ maxReadConcurrentRatio: 1
454
+ cpuRatio: 10 # ratio used to estimate read task cpu usage.
455
+ maxTimestampLag: 86400
456
+ scheduleReadPolicy:
457
+ # fifo: A FIFO queue support the schedule.
458
+ # user-task-polling:
459
+ # The user's tasks will be polled one by one and scheduled.
460
+ # Scheduling is fair on task granularity.
461
+ # The policy is based on the username for authentication.
462
+ # And an empty username is considered the same user.
463
+ # When there are no multi-users, the policy decay into FIFO"
464
+ name: fifo
465
+ taskQueueExpire: 60 # Control how long (many seconds) that queue retains since queue is empty
466
+ enableCrossUserGrouping: false # Enable Cross user grouping when using user-task-polling policy. (Disable it if user's task can not merge each other)
467
+ maxPendingTaskPerUser: 1024 # Max pending task per user in scheduler
468
+ levelZeroForwardPolicy: RemoteLoad # delegator level zero deletion forward policy, possible option["FilterByBF", "RemoteLoad"]
469
+ streamingDeltaForwardPolicy: FilterByBF # delegator streaming deletion forward policy, possible option["FilterByBF", "Direct"]
470
+ dataSync:
471
+ flowGraph:
472
+ maxQueueLength: 16 # The maximum size of task queue cache in flow graph in query node.
473
+ maxParallelism: 1024 # Maximum number of tasks executed in parallel in the flowgraph
474
+ enableSegmentPrune: false # use partition stats to prune data in search/query on shard delegator
475
+ queryStreamBatchSize: 4194304 # return min batch size of stream query
476
+ queryStreamMaxBatchSize: 134217728 # return max batch size of stream query
477
+ bloomFilterApplyParallelFactor: 4 # parallel factor when to apply pk to bloom filter, default to 4*CPU_CORE_NUM
478
+ workerPooling:
479
+ size: 10 # the size for worker querynode client pool
480
+ ip: # TCP/IP address of queryNode. If not specified, use the first unicastable address
481
+ port: 21123 # TCP port of queryNode
482
+ grpc:
483
+ serverMaxSendSize: 536870912 # The maximum size of each RPC request that the queryNode can send, unit: byte
484
+ serverMaxRecvSize: 268435456 # The maximum size of each RPC request that the queryNode can receive, unit: byte
485
+ clientMaxSendSize: 268435456 # The maximum size of each RPC request that the clients on queryNode can send, unit: byte
486
+ clientMaxRecvSize: 536870912 # The maximum size of each RPC request that the clients on queryNode can receive, unit: byte
487
+
488
+ indexCoord:
489
+ bindIndexNodeMode:
490
+ enable: false
491
+ address: localhost:22930
492
+ withCred: false
493
+ nodeID: 0
494
+ segment:
495
+ minSegmentNumRowsToEnableIndex: 1024 # It's a threshold. When the segment num rows is less than this value, the segment will not be indexed
496
+
497
+ indexNode:
498
+ scheduler:
499
+ buildParallel: 1
500
+ enableDisk: true # enable index node build disk vector index
501
+ maxDiskUsagePercentage: 95
502
+ ip: # TCP/IP address of indexNode. If not specified, use the first unicastable address
503
+ port: 21121 # TCP port of indexNode
504
+ grpc:
505
+ serverMaxSendSize: 536870912 # The maximum size of each RPC request that the indexNode can send, unit: byte
506
+ serverMaxRecvSize: 268435456 # The maximum size of each RPC request that the indexNode can receive, unit: byte
507
+ clientMaxSendSize: 268435456 # The maximum size of each RPC request that the clients on indexNode can send, unit: byte
508
+ clientMaxRecvSize: 536870912 # The maximum size of each RPC request that the clients on indexNode can receive, unit: byte
509
+
510
+ dataCoord:
511
+ channel:
512
+ watchTimeoutInterval: 300 # Timeout on watching channels (in seconds). Datanode tickler update watch progress will reset timeout timer.
513
+ legacyVersionWithoutRPCWatch: 2.4.1 # Datanodes <= this version are considered as legacy nodes, which doesn't have rpc based watch(). This is only used during rolling upgrade where legacy nodes won't get new channels
514
+ balanceSilentDuration: 300 # The duration after which the channel manager start background channel balancing
515
+ balanceInterval: 360 # The interval with which the channel manager check dml channel balance status
516
+ checkInterval: 1 # The interval in seconds with which the channel manager advances channel states
517
+ notifyChannelOperationTimeout: 5 # Timeout notifing channel operations (in seconds).
518
+ segment:
519
+ maxSize: 1024 # The maximum size of a segment, unit: MB. datacoord.segment.maxSize and datacoord.segment.sealProportion together determine if a segment can be sealed.
520
+ diskSegmentMaxSize: 2048 # Maximun size of a segment in MB for collection which has Disk index
521
+ sealProportion: 0.12 # The minimum proportion to datacoord.segment.maxSize to seal a segment. datacoord.segment.maxSize and datacoord.segment.sealProportion together determine if a segment can be sealed.
522
+ sealProportionJitter: 0.1 # segment seal proportion jitter ratio, default value 0.1(10%), if seal proportion is 12%, with jitter=0.1, the actuall applied ratio will be 10.8~12%
523
+ assignmentExpiration: 2000 # Expiration time of the segment assignment, unit: ms
524
+ allocLatestExpireAttempt: 200 # The time attempting to alloc latest lastExpire from rootCoord after restart
525
+ maxLife: 86400 # The max lifetime of segment in seconds, 24*60*60
526
+ # If a segment didn't accept dml records in maxIdleTime and the size of segment is greater than
527
+ # minSizeFromIdleToSealed, Milvus will automatically seal it.
528
+ # The max idle time of segment in seconds, 10*60.
529
+ maxIdleTime: 600
530
+ minSizeFromIdleToSealed: 16 # The min size in MB of segment which can be idle from sealed.
531
+ # The max number of binlog (which is equal to the binlog file num of primary key) for one segment,
532
+ # the segment will be sealed if the number of binlog file reaches to max value.
533
+ maxBinlogFileNumber: 32
534
+ smallProportion: 0.5 # The segment is considered as "small segment" when its # of rows is smaller than
535
+ # (smallProportion * segment max # of rows).
536
+ # A compaction will happen on small segments if the segment after compaction will have
537
+ compactableProportion: 0.85
538
+ # over (compactableProportion * segment max # of rows) rows.
539
+ # MUST BE GREATER THAN OR EQUAL TO <smallProportion>!!!
540
+ # During compaction, the size of segment # of rows is able to exceed segment max # of rows by (expansionRate-1) * 100%.
541
+ expansionRate: 1.25
542
+ sealPolicy:
543
+ channel:
544
+ # The size threshold in MB, if the total size of growing segments of each shard
545
+ # exceeds this threshold, the largest growing segment will be sealed.
546
+ growingSegmentsMemSize: 4096
547
+ autoUpgradeSegmentIndex: false # whether auto upgrade segment index to index engine's version
548
+ segmentFlushInterval: 2 # the minimal interval duration(unit: Seconds) between flusing operation on same segment
549
+ # Switch value to control if to enable segment compaction.
550
+ # Compaction merges small-size segments into a large segment, and clears the entities deleted beyond the rentention duration of Time Travel.
551
+ enableCompaction: true
552
+ compaction:
553
+ # Switch value to control if to enable automatic segment compaction during which data coord locates and merges compactable segments in the background.
554
+ # This configuration takes effect only when dataCoord.enableCompaction is set as true.
555
+ enableAutoCompaction: true
556
+ indexBasedCompaction: true
557
+ # compaction task prioritizer, options: [default, level, mix].
558
+ # default is FIFO.
559
+ # level is prioritized by level: L0 compactions first, then mix compactions, then clustering compactions.
560
+ # mix is prioritized by level: mix compactions first, then L0 compactions, then clustering compactions.
561
+ taskPrioritizer: default
562
+ taskQueueCapacity: 256 # compaction task queue size
563
+ rpcTimeout: 10
564
+ maxParallelTaskNum: 10
565
+ dropTolerance: 86400 # Compaction task will be cleaned after finish longer than this time(in seconds)
566
+ gcInterval: 1800 # The time interval in seconds for compaction gc
567
+ mix:
568
+ triggerInterval: 60 # The time interval in seconds to trigger mix compaction
569
+ levelzero:
570
+ triggerInterval: 10 # The time interval in seconds for trigger L0 compaction
571
+ forceTrigger:
572
+ minSize: 8388608 # The minmum size in bytes to force trigger a LevelZero Compaction, default as 8MB
573
+ maxSize: 67108864 # The maxmum size in bytes to force trigger a LevelZero Compaction, default as 64MB
574
+ deltalogMinNum: 10 # The minimum number of deltalog files to force trigger a LevelZero Compaction
575
+ deltalogMaxNum: 30 # The maxmum number of deltalog files to force trigger a LevelZero Compaction, default as 30
576
+ single:
577
+ ratio:
578
+ threshold: 0.2 # The ratio threshold of a segment to trigger a single compaction, default as 0.2
579
+ deltalog:
580
+ maxsize: 16777216 # The deltalog size of a segment to trigger a single compaction, default as 16MB
581
+ maxnum: 200 # The deltalog count of a segment to trigger a compaction, default as 200
582
+ expiredlog:
583
+ maxsize: 10485760 # The expired log size of a segment to trigger a compaction, default as 10MB
584
+ clustering:
585
+ enable: true # Enable clustering compaction
586
+ autoEnable: false # Enable auto clustering compaction
587
+ triggerInterval: 600 # clustering compaction trigger interval in seconds
588
+ minInterval: 3600 # The minimum interval between clustering compaction executions of one collection, to avoid redundant compaction
589
+ maxInterval: 259200 # If a collection haven't been clustering compacted for longer than maxInterval, force compact
590
+ newDataSizeThreshold: 512m # If new data size is large than newDataSizeThreshold, execute clustering compaction
591
+ preferSegmentSizeRatio: 0.8
592
+ maxSegmentSizeRatio: 1
593
+ maxTrainSizeRatio: 0.8 # max data size ratio in Kmeans train, if larger than it, will down sampling to meet this limit
594
+ maxCentroidsNum: 10240 # maximum centroids number in Kmeans train
595
+ minCentroidsNum: 16 # minimum centroids number in Kmeans train
596
+ minClusterSizeRatio: 0.01 # minimum cluster size / avg size in Kmeans train
597
+ maxClusterSizeRatio: 10 # maximum cluster size / avg size in Kmeans train
598
+ maxClusterSize: 5g # maximum cluster size in Kmeans train
599
+ syncSegmentsInterval: 300 # The time interval for regularly syncing segments
600
+ enableGarbageCollection: true # Switch value to control if to enable garbage collection to clear the discarded data in MinIO or S3 service.
601
+ gc:
602
+ interval: 3600 # The interval at which data coord performs garbage collection, unit: second.
603
+ missingTolerance: 86400 # The retention duration of the unrecorded binary log (binlog) files. Setting a reasonably large value for this parameter avoids erroneously deleting the newly created binlog files that lack metadata. Unit: second.
604
+ dropTolerance: 10800 # The retention duration of the binlog files of the deleted segments before they are cleared, unit: second.
605
+ removeConcurrent: 32 # number of concurrent goroutines to remove dropped s3 objects
606
+ scanInterval: 168 # orphan file (file on oss but has not been registered on meta) on object storage garbage collection scanning interval in hours
607
+ enableActiveStandby: false
608
+ brokerTimeout: 5000 # 5000ms, dataCoord broker rpc timeout
609
+ autoBalance: true # Enable auto balance
610
+ checkAutoBalanceConfigInterval: 10 # the interval of check auto balance config
611
+ import:
612
+ filesPerPreImportTask: 2 # The maximum number of files allowed per pre-import task.
613
+ taskRetention: 10800 # The retention period in seconds for tasks in the Completed or Failed state.
614
+ maxSizeInMBPerImportTask: 6144 # To prevent generating of small segments, we will re-group imported files. This parameter represents the sum of file sizes in each group (each ImportTask).
615
+ scheduleInterval: 2 # The interval for scheduling import, measured in seconds.
616
+ checkIntervalHigh: 2 # The interval for checking import, measured in seconds, is set to a high frequency for the import checker.
617
+ checkIntervalLow: 120 # The interval for checking import, measured in seconds, is set to a low frequency for the import checker.
618
+ maxImportFileNumPerReq: 1024 # The maximum number of files allowed per single import request.
619
+ maxImportJobNum: 1024 # Maximum number of import jobs that are executing or pending.
620
+ waitForIndex: true # Indicates whether the import operation waits for the completion of index building.
621
+ gracefulStopTimeout: 5 # seconds. force stop node without graceful stop
622
+ slot:
623
+ clusteringCompactionUsage: 16 # slot usage of clustering compaction job.
624
+ mixCompactionUsage: 8 # slot usage of mix compaction job.
625
+ l0DeleteCompactionUsage: 8 # slot usage of l0 compaction job.
626
+ ip: # TCP/IP address of dataCoord. If not specified, use the first unicastable address
627
+ port: 13333 # TCP port of dataCoord
628
+ grpc:
629
+ serverMaxSendSize: 536870912 # The maximum size of each RPC request that the dataCoord can send, unit: byte
630
+ serverMaxRecvSize: 268435456 # The maximum size of each RPC request that the dataCoord can receive, unit: byte
631
+ clientMaxSendSize: 268435456 # The maximum size of each RPC request that the clients on dataCoord can send, unit: byte
632
+ clientMaxRecvSize: 536870912 # The maximum size of each RPC request that the clients on dataCoord can receive, unit: byte
633
+
634
+ dataNode:
635
+ dataSync:
636
+ flowGraph:
637
+ maxQueueLength: 16 # Maximum length of task queue in flowgraph
638
+ maxParallelism: 1024 # Maximum number of tasks executed in parallel in the flowgraph
639
+ maxParallelSyncMgrTasks: 256 # The max concurrent sync task number of datanode sync mgr globally
640
+ skipMode:
641
+ enable: true # Support skip some timetick message to reduce CPU usage
642
+ skipNum: 4 # Consume one for every n records skipped
643
+ coldTime: 60 # Turn on skip mode after there are only timetick msg for x seconds
644
+ segment:
645
+ # The maximum size of each binlog file in a segment buffered in memory. Binlog files whose size exceeds this value are then flushed to MinIO or S3 service.
646
+ # Unit: Byte
647
+ # Setting this parameter too small causes the system to store a small amount of data too frequently. Setting it too large increases the system's demand for memory.
648
+ insertBufSize: 16777216
649
+ deleteBufBytes: 16777216 # Max buffer size in bytes to flush del for a single channel, default as 16MB
650
+ syncPeriod: 600 # The period to sync segments if buffer is not empty.
651
+ memory:
652
+ forceSyncEnable: true # Set true to force sync if memory usage is too high
653
+ forceSyncSegmentNum: 1 # number of segments to sync, segments with top largest buffer will be synced.
654
+ checkInterval: 3000 # the interal to check datanode memory usage, in milliseconds
655
+ forceSyncWatermark: 0.5 # memory watermark for standalone, upon reaching this watermark, segments will be synced.
656
+ timetick:
657
+ interval: 500
658
+ channel:
659
+ # specify the size of global work pool of all channels
660
+ # if this parameter <= 0, will set it as the maximum number of CPUs that can be executing
661
+ # suggest to set it bigger on large collection numbers to avoid blocking
662
+ workPoolSize: -1
663
+ # specify the size of global work pool for channel checkpoint updating
664
+ # if this parameter <= 0, will set it as 10
665
+ updateChannelCheckpointMaxParallel: 10
666
+ updateChannelCheckpointInterval: 60 # the interval duration(in seconds) for datanode to update channel checkpoint of each channel
667
+ updateChannelCheckpointRPCTimeout: 20 # timeout in seconds for UpdateChannelCheckpoint RPC call
668
+ maxChannelCheckpointsPerPRC: 128 # The maximum number of channel checkpoints per UpdateChannelCheckpoint RPC.
669
+ channelCheckpointUpdateTickInSeconds: 10 # The frequency, in seconds, at which the channel checkpoint updater executes updates.
670
+ import:
671
+ maxConcurrentTaskNum: 16 # The maximum number of import/pre-import tasks allowed to run concurrently on a datanode.
672
+ maxImportFileSizeInGB: 16 # The maximum file size (in GB) for an import file, where an import file refers to either a Row-Based file or a set of Column-Based files.
673
+ readBufferSizeInMB: 16 # The data block size (in MB) read from chunk manager by the datanode during import.
674
+ maxTaskSlotNum: 16 # The maximum number of slots occupied by each import/pre-import task.
675
+ compaction:
676
+ levelZeroBatchMemoryRatio: 0.5 # The minimal memory ratio of free memory for level zero compaction executing in batch mode
677
+ levelZeroMaxBatchSize: -1 # Max batch size refers to the max number of L1/L2 segments in a batch when executing L0 compaction. Default to -1, any value that is less than 1 means no limit. Valid range: >= 1.
678
+ gracefulStopTimeout: 1800 # seconds. force stop node without graceful stop
679
+ slot:
680
+ slotCap: 16 # The maximum number of tasks(e.g. compaction, importing) allowed to run concurrently on a datanode
681
+ clusteringCompaction:
682
+ memoryBufferRatio: 0.1 # The ratio of memory buffer of clustering compaction. Data larger than threshold will be flushed to storage.
683
+ workPoolSize: 8 # worker pool size for one clustering compaction job.
684
+ bloomFilterApplyParallelFactor: 4 # parallel factor when to apply pk to bloom filter, default to 4*CPU_CORE_NUM
685
+ storage:
686
+ deltalog: json # deltalog format, options: [json, parquet]
687
+ ip: # TCP/IP address of dataNode. If not specified, use the first unicastable address
688
+ port: 21124 # TCP port of dataNode
689
+ grpc:
690
+ serverMaxSendSize: 536870912 # The maximum size of each RPC request that the dataNode can send, unit: byte
691
+ serverMaxRecvSize: 268435456 # The maximum size of each RPC request that the dataNode can receive, unit: byte
692
+ clientMaxSendSize: 268435456 # The maximum size of each RPC request that the clients on dataNode can send, unit: byte
693
+ clientMaxRecvSize: 536870912 # The maximum size of each RPC request that the clients on dataNode can receive, unit: byte
694
+
695
+ # This topic introduces the message channel-related configurations of Milvus.
696
+ msgChannel:
697
+ chanNamePrefix:
698
+ # Root name prefix of the channel when a message channel is created.
699
+ # It is recommended to change this parameter before starting Milvus for the first time.
700
+ # To share a Pulsar instance among multiple Milvus instances, consider changing this to a name rather than the default one for each Milvus instance before you start them.
701
+ cluster: by-dev
702
+ # Sub-name prefix of the message channel where the root coord publishes time tick messages.
703
+ # The complete channel name prefix is ${msgChannel.chanNamePrefix.cluster}-${msgChannel.chanNamePrefix.rootCoordTimeTick}
704
+ # Caution: Changing this parameter after using Milvus for a period of time will affect your access to old data.
705
+ # It is recommended to change this parameter before starting Milvus for the first time.
706
+ rootCoordTimeTick: rootcoord-timetick
707
+ # Sub-name prefix of the message channel where the root coord publishes its own statistics messages.
708
+ # The complete channel name prefix is ${msgChannel.chanNamePrefix.cluster}-${msgChannel.chanNamePrefix.rootCoordStatistics}
709
+ # Caution: Changing this parameter after using Milvus for a period of time will affect your access to old data.
710
+ # It is recommended to change this parameter before starting Milvus for the first time.
711
+ rootCoordStatistics: rootcoord-statistics
712
+ # Sub-name prefix of the message channel where the root coord publishes Data Manipulation Language (DML) messages.
713
+ # The complete channel name prefix is ${msgChannel.chanNamePrefix.cluster}-${msgChannel.chanNamePrefix.rootCoordDml}
714
+ # Caution: Changing this parameter after using Milvus for a period of time will affect your access to old data.
715
+ # It is recommended to change this parameter before starting Milvus for the first time.
716
+ rootCoordDml: rootcoord-dml
717
+ replicateMsg: replicate-msg
718
+ # Sub-name prefix of the message channel where the query node publishes time tick messages.
719
+ # The complete channel name prefix is ${msgChannel.chanNamePrefix.cluster}-${msgChannel.chanNamePrefix.queryTimeTick}
720
+ # Caution: Changing this parameter after using Milvus for a period of time will affect your access to old data.
721
+ # It is recommended to change this parameter before starting Milvus for the first time.
722
+ queryTimeTick: queryTimeTick
723
+ # Sub-name prefix of the message channel where the data coord publishes time tick messages.
724
+ # The complete channel name prefix is ${msgChannel.chanNamePrefix.cluster}-${msgChannel.chanNamePrefix.dataCoordTimeTick}
725
+ # Caution: Changing this parameter after using Milvus for a period of time will affect your access to old data.
726
+ # It is recommended to change this parameter before starting Milvus for the first time.
727
+ dataCoordTimeTick: datacoord-timetick-channel
728
+ # Sub-name prefix of the message channel where the data coord publishes segment information messages.
729
+ # The complete channel name prefix is ${msgChannel.chanNamePrefix.cluster}-${msgChannel.chanNamePrefix.dataCoordSegmentInfo}
730
+ # Caution: Changing this parameter after using Milvus for a period of time will affect your access to old data.
731
+ # It is recommended to change this parameter before starting Milvus for the first time.
732
+ dataCoordSegmentInfo: segment-info-channel
733
+ subNamePrefix:
734
+ # Subscription name prefix of the data coord.
735
+ # Caution: Changing this parameter after using Milvus for a period of time will affect your access to old data.
736
+ # It is recommended to change this parameter before starting Milvus for the first time.
737
+ dataCoordSubNamePrefix: dataCoord
738
+ # Subscription name prefix of the data node.
739
+ # Caution: Changing this parameter after using Milvus for a period of time will affect your access to old data.
740
+ # It is recommended to change this parameter before starting Milvus for the first time.
741
+ dataNodeSubNamePrefix: dataNode
742
+
743
+ # Configures the system log output.
744
+ log:
745
+ # Milvus log level. Option: debug, info, warn, error, panic, and fatal.
746
+ # It is recommended to use debug level under test and development environments, and info level in production environment.
747
+ level: info
748
+ file:
749
+ # Root path to the log files.
750
+ # The default value is set empty, indicating to output log files to standard output (stdout) and standard error (stderr).
751
+ # If this parameter is set to a valid local path, Milvus writes and stores log files in this path.
752
+ # Set this parameter as the path that you have permission to write.
753
+ rootPath:
754
+ maxSize: 300 # The maximum size of a log file, unit: MB.
755
+ maxAge: 10 # The maximum retention time before a log file is automatically cleared, unit: day. The minimum value is 1.
756
+ maxBackups: 20 # The maximum number of log files to back up, unit: day. The minimum value is 1.
757
+ format: text # Milvus log format. Option: text and JSON
758
+ stdout: true # Stdout enable or not
759
+
760
+ grpc:
761
+ log:
762
+ level: WARNING
763
+ gracefulStopTimeout: 10 # second, time to wait graceful stop finish
764
+ client:
765
+ compressionEnabled: false
766
+ dialTimeout: 200
767
+ keepAliveTime: 10000
768
+ keepAliveTimeout: 20000
769
+ maxMaxAttempts: 10
770
+ initialBackoff: 0.2
771
+ maxBackoff: 10
772
+ backoffMultiplier: 2
773
+ minResetInterval: 1000
774
+ maxCancelError: 32
775
+ minSessionCheckInterval: 200
776
+
777
+ # Configure the proxy tls enable.
778
+ tls:
779
+ serverPemPath: configs/cert/server.pem
780
+ serverKeyPath: configs/cert/server.key
781
+ caPemPath: configs/cert/ca.pem
782
+
783
+ common:
784
+ defaultPartitionName: _default # Name of the default partition when a collection is created
785
+ defaultIndexName: _default_idx # Name of the index when it is created with name unspecified
786
+ entityExpiration: -1 # Entity expiration in seconds, CAUTION -1 means never expire
787
+ indexSliceSize: 16 # Index slice size in MB
788
+ threadCoreCoefficient:
789
+ highPriority: 10 # This parameter specify how many times the number of threads is the number of cores in high priority pool
790
+ middlePriority: 5 # This parameter specify how many times the number of threads is the number of cores in middle priority pool
791
+ lowPriority: 1 # This parameter specify how many times the number of threads is the number of cores in low priority pool
792
+ buildIndexThreadPoolRatio: 0.75
793
+ DiskIndex:
794
+ MaxDegree: 56
795
+ SearchListSize: 100
796
+ PQCodeBudgetGBRatio: 0.125
797
+ BuildNumThreadsRatio: 1
798
+ SearchCacheBudgetGBRatio: 0.1
799
+ LoadNumThreadRatio: 8
800
+ BeamWidthRatio: 4
801
+ gracefulTime: 5000 # milliseconds. it represents the interval (in ms) by which the request arrival time needs to be subtracted in the case of Bounded Consistency.
802
+ gracefulStopTimeout: 1800 # seconds. it will force quit the server if the graceful stop process is not completed during this time.
803
+ storageType: remote # please adjust in embedded Milvus: local, available values are [local, remote, opendal], value minio is deprecated, use remote instead
804
+ # Default value: auto
805
+ # Valid values: [auto, avx512, avx2, avx, sse4_2]
806
+ # This configuration is only used by querynode and indexnode, it selects CPU instruction set for Searching and Index-building.
807
+ simdType: auto
808
+ security:
809
+ authorizationEnabled: false
810
+ # The superusers will ignore some system check processes,
811
+ # like the old password verification when updating the credential
812
+ superUsers:
813
+ defaultRootPassword: Milvus # default password for root user
814
+ tlsMode: 0
815
+ session:
816
+ ttl: 30 # ttl value when session granting a lease to register service
817
+ retryTimes: 30 # retry times when session sending etcd requests
818
+ locks:
819
+ metrics:
820
+ enable: false # whether gather statistics for metrics locks
821
+ threshold:
822
+ info: 500 # minimum milliseconds for printing durations in info level
823
+ warn: 1000 # minimum milliseconds for printing durations in warn level
824
+ storage:
825
+ scheme: s3
826
+ enablev2: false
827
+ # Whether to disable the internal time messaging mechanism for the system.
828
+ # If disabled (set to false), the system will not allow DML operations, including insertion, deletion, queries, and searches.
829
+ # This helps Milvus-CDC synchronize incremental data
830
+ ttMsgEnabled: true
831
+ traceLogMode: 0 # trace request info
832
+ bloomFilterSize: 100000 # bloom filter initial size
833
+ bloomFilterType: BlockedBloomFilter # bloom filter type, support BasicBloomFilter and BlockedBloomFilter
834
+ maxBloomFalsePositive: 0.001 # max false positive rate for bloom filter
835
+ bloomFilterApplyBatchSize: 1000 # batch size when to apply pk to bloom filter
836
+ usePartitionKeyAsClusteringKey: false # if true, do clustering compaction and segment prune on partition key field
837
+ useVectorAsClusteringKey: false # if true, do clustering compaction and segment prune on vector field
838
+ enableVectorClusteringKey: false # if true, enable vector clustering key and vector clustering compaction
839
+
840
+ # QuotaConfig, configurations of Milvus quota and limits.
841
+ # By default, we enable:
842
+ # 1. TT protection;
843
+ # 2. Memory protection.
844
+ # 3. Disk quota protection.
845
+ # You can enable:
846
+ # 1. DML throughput limitation;
847
+ # 2. DDL, DQL qps/rps limitation;
848
+ # 3. DQL Queue length/latency protection;
849
+ # 4. DQL result rate protection;
850
+ # If necessary, you can also manually force to deny RW requests.
851
+ quotaAndLimits:
852
+ enabled: true # `true` to enable quota and limits, `false` to disable.
853
+ # quotaCenterCollectInterval is the time interval that quotaCenter
854
+ # collects metrics from Proxies, Query cluster and Data cluster.
855
+ # seconds, (0 ~ 65536)
856
+ quotaCenterCollectInterval: 3
857
+ limits:
858
+ allocRetryTimes: 15 # retry times when delete alloc forward data from rate limit failed
859
+ allocWaitInterval: 1000 # retry wait duration when delete alloc forward data rate failed, in millisecond
860
+ complexDeleteLimitEnable: false # whether complex delete check forward data by limiter
861
+ maxCollectionNum: 65536
862
+ maxCollectionNumPerDB: 65536 # Maximum number of collections per database.
863
+ maxInsertSize: -1 # maximum size of a single insert request, in bytes, -1 means no limit
864
+ maxResourceGroupNumOfQueryNode: 1024 # maximum number of resource groups of query nodes
865
+ maxGroupSize: 10 # maximum size for one single group when doing search group by
866
+ ddl:
867
+ enabled: false # Whether DDL request throttling is enabled.
868
+ # Maximum number of collection-related DDL requests per second.
869
+ # Setting this item to 10 indicates that Milvus processes no more than 10 collection-related DDL requests per second, including collection creation requests, collection drop requests, collection load requests, and collection release requests.
870
+ # To use this setting, set quotaAndLimits.ddl.enabled to true at the same time.
871
+ collectionRate: -1
872
+ # Maximum number of partition-related DDL requests per second.
873
+ # Setting this item to 10 indicates that Milvus processes no more than 10 partition-related requests per second, including partition creation requests, partition drop requests, partition load requests, and partition release requests.
874
+ # To use this setting, set quotaAndLimits.ddl.enabled to true at the same time.
875
+ partitionRate: -1
876
+ db:
877
+ collectionRate: -1 # qps of db level , default no limit, rate for CreateCollection, DropCollection, LoadCollection, ReleaseCollection
878
+ partitionRate: -1 # qps of db level, default no limit, rate for CreatePartition, DropPartition, LoadPartition, ReleasePartition
879
+ indexRate:
880
+ enabled: false # Whether index-related request throttling is enabled.
881
+ # Maximum number of index-related requests per second.
882
+ # Setting this item to 10 indicates that Milvus processes no more than 10 partition-related requests per second, including index creation requests and index drop requests.
883
+ # To use this setting, set quotaAndLimits.indexRate.enabled to true at the same time.
884
+ max: -1
885
+ db:
886
+ max: -1 # qps of db level, default no limit, rate for CreateIndex, DropIndex
887
+ flushRate:
888
+ enabled: true # Whether flush request throttling is enabled.
889
+ # Maximum number of flush requests per second.
890
+ # Setting this item to 10 indicates that Milvus processes no more than 10 flush requests per second.
891
+ # To use this setting, set quotaAndLimits.flushRate.enabled to true at the same time.
892
+ max: -1
893
+ collection:
894
+ max: 0.1 # qps, default no limit, rate for flush at collection level.
895
+ db:
896
+ max: -1 # qps of db level, default no limit, rate for flush
897
+ compactionRate:
898
+ enabled: false # Whether manual compaction request throttling is enabled.
899
+ # Maximum number of manual-compaction requests per second.
900
+ # Setting this item to 10 indicates that Milvus processes no more than 10 manual-compaction requests per second.
901
+ # To use this setting, set quotaAndLimits.compaction.enabled to true at the same time.
902
+ max: -1
903
+ db:
904
+ max: -1 # qps of db level, default no limit, rate for manualCompaction
905
+ dml:
906
+ enabled: false # Whether DML request throttling is enabled.
907
+ insertRate:
908
+ # Highest data insertion rate per second.
909
+ # Setting this item to 5 indicates that Milvus only allows data insertion at the rate of 5 MB/s.
910
+ # To use this setting, set quotaAndLimits.dml.enabled to true at the same time.
911
+ max: -1
912
+ db:
913
+ max: -1 # MB/s, default no limit
914
+ collection:
915
+ # Highest data insertion rate per collection per second.
916
+ # Setting this item to 5 indicates that Milvus only allows data insertion to any collection at the rate of 5 MB/s.
917
+ # To use this setting, set quotaAndLimits.dml.enabled to true at the same time.
918
+ max: -1
919
+ partition:
920
+ max: -1 # MB/s, default no limit
921
+ upsertRate:
922
+ max: -1 # MB/s, default no limit
923
+ db:
924
+ max: -1 # MB/s, default no limit
925
+ collection:
926
+ max: -1 # MB/s, default no limit
927
+ partition:
928
+ max: -1 # MB/s, default no limit
929
+ deleteRate:
930
+ # Highest data deletion rate per second.
931
+ # Setting this item to 0.1 indicates that Milvus only allows data deletion at the rate of 0.1 MB/s.
932
+ # To use this setting, set quotaAndLimits.dml.enabled to true at the same time.
933
+ max: -1
934
+ db:
935
+ max: -1 # MB/s, default no limit
936
+ collection:
937
+ # Highest data deletion rate per second.
938
+ # Setting this item to 0.1 indicates that Milvus only allows data deletion from any collection at the rate of 0.1 MB/s.
939
+ # To use this setting, set quotaAndLimits.dml.enabled to true at the same time.
940
+ max: -1
941
+ partition:
942
+ max: -1 # MB/s, default no limit
943
+ bulkLoadRate:
944
+ max: -1 # MB/s, default no limit, not support yet. TODO: limit bulkLoad rate
945
+ db:
946
+ max: -1 # MB/s, default no limit, not support yet. TODO: limit db bulkLoad rate
947
+ collection:
948
+ max: -1 # MB/s, default no limit, not support yet. TODO: limit collection bulkLoad rate
949
+ partition:
950
+ max: -1 # MB/s, default no limit, not support yet. TODO: limit partition bulkLoad rate
951
+ dql:
952
+ enabled: false # Whether DQL request throttling is enabled.
953
+ searchRate:
954
+ # Maximum number of vectors to search per second.
955
+ # Setting this item to 100 indicates that Milvus only allows searching 100 vectors per second no matter whether these 100 vectors are all in one search or scattered across multiple searches.
956
+ # To use this setting, set quotaAndLimits.dql.enabled to true at the same time.
957
+ max: -1
958
+ db:
959
+ max: -1 # vps (vectors per second), default no limit
960
+ collection:
961
+ # Maximum number of vectors to search per collection per second.
962
+ # Setting this item to 100 indicates that Milvus only allows searching 100 vectors per second per collection no matter whether these 100 vectors are all in one search or scattered across multiple searches.
963
+ # To use this setting, set quotaAndLimits.dql.enabled to true at the same time.
964
+ max: -1
965
+ partition:
966
+ max: -1 # vps (vectors per second), default no limit
967
+ queryRate:
968
+ # Maximum number of queries per second.
969
+ # Setting this item to 100 indicates that Milvus only allows 100 queries per second.
970
+ # To use this setting, set quotaAndLimits.dql.enabled to true at the same time.
971
+ max: -1
972
+ db:
973
+ max: -1 # qps, default no limit
974
+ collection:
975
+ # Maximum number of queries per collection per second.
976
+ # Setting this item to 100 indicates that Milvus only allows 100 queries per collection per second.
977
+ # To use this setting, set quotaAndLimits.dql.enabled to true at the same time.
978
+ max: -1
979
+ partition:
980
+ max: -1 # qps, default no limit
981
+ limitWriting:
982
+ # forceDeny false means dml requests are allowed (except for some
983
+ # specific conditions, such as memory of nodes to water marker), true means always reject all dml requests.
984
+ forceDeny: false
985
+ ttProtection:
986
+ enabled: false
987
+ # maxTimeTickDelay indicates the backpressure for DML Operations.
988
+ # DML rates would be reduced according to the ratio of time tick delay to maxTimeTickDelay,
989
+ # if time tick delay is greater than maxTimeTickDelay, all DML requests would be rejected.
990
+ # seconds
991
+ maxTimeTickDelay: 300
992
+ memProtection:
993
+ # When memory usage > memoryHighWaterLevel, all dml requests would be rejected;
994
+ # When memoryLowWaterLevel < memory usage < memoryHighWaterLevel, reduce the dml rate;
995
+ # When memory usage < memoryLowWaterLevel, no action.
996
+ enabled: true
997
+ dataNodeMemoryLowWaterLevel: 0.85 # (0, 1], memoryLowWaterLevel in DataNodes
998
+ dataNodeMemoryHighWaterLevel: 0.95 # (0, 1], memoryHighWaterLevel in DataNodes
999
+ queryNodeMemoryLowWaterLevel: 0.85 # (0, 1], memoryLowWaterLevel in QueryNodes
1000
+ queryNodeMemoryHighWaterLevel: 0.95 # (0, 1], memoryHighWaterLevel in QueryNodes
1001
+ growingSegmentsSizeProtection:
1002
+ # No action will be taken if the growing segments size is less than the low watermark.
1003
+ # When the growing segments size exceeds the low watermark, the dml rate will be reduced,
1004
+ # but the rate will not be lower than minRateRatio * dmlRate.
1005
+ enabled: false
1006
+ minRateRatio: 0.5
1007
+ lowWaterLevel: 0.2
1008
+ highWaterLevel: 0.4
1009
+ diskProtection:
1010
+ enabled: true # When the total file size of object storage is greater than `diskQuota`, all dml requests would be rejected;
1011
+ diskQuota: -1 # MB, (0, +inf), default no limit
1012
+ diskQuotaPerDB: -1 # MB, (0, +inf), default no limit
1013
+ diskQuotaPerCollection: -1 # MB, (0, +inf), default no limit
1014
+ diskQuotaPerPartition: -1 # MB, (0, +inf), default no limit
1015
+ l0SegmentsRowCountProtection:
1016
+ enabled: false # switch to enable l0 segment row count quota
1017
+ lowWaterLevel: 30000000 # l0 segment row count quota, low water level
1018
+ highWaterLevel: 50000000 # l0 segment row count quota, high water level
1019
+ deleteBufferRowCountProtection:
1020
+ enabled: false # switch to enable delete buffer row count quota
1021
+ lowWaterLevel: 32768 # delete buffer row count quota, low water level
1022
+ highWaterLevel: 65536 # delete buffer row count quota, high water level
1023
+ deleteBufferSizeProtection:
1024
+ enabled: false # switch to enable delete buffer size quota
1025
+ lowWaterLevel: 134217728 # delete buffer size quota, low water level
1026
+ highWaterLevel: 268435456 # delete buffer size quota, high water level
1027
+ limitReading:
1028
+ # forceDeny false means dql requests are allowed (except for some
1029
+ # specific conditions, such as collection has been dropped), true means always reject all dql requests.
1030
+ forceDeny: false
1031
+
1032
+ trace:
1033
+ # trace exporter type, default is stdout,
1034
+ # optional values: ['noop','stdout', 'jaeger', 'otlp']
1035
+ exporter: noop
1036
+ # fraction of traceID based sampler,
1037
+ # optional values: [0, 1]
1038
+ # Fractions >= 1 will always sample. Fractions < 0 are treated as zero.
1039
+ sampleFraction: 0
1040
+ jaeger:
1041
+ url: # when exporter is jaeger should set the jaeger's URL
1042
+ otlp:
1043
+ endpoint: # example: "127.0.0.1:4317" for grpc, "127.0.0.1:4318" for http
1044
+ method: # otlp export method, acceptable values: ["grpc", "http"], using "grpc" by default
1045
+ secure: true
1046
+ initTimeoutSeconds: 10 # segcore initialization timeout in seconds, preventing otlp grpc hangs forever
1047
+
1048
+ #when using GPU indexing, Milvus will utilize a memory pool to avoid frequent memory allocation and deallocation.
1049
+ #here, you can set the size of the memory occupied by the memory pool, with the unit being MB.
1050
+ #note that there is a possibility of Milvus crashing when the actual memory demand exceeds the value set by maxMemSize.
1051
+ #if initMemSize and MaxMemSize both set zero,
1052
+ #milvus will automatically initialize half of the available GPU memory,
1053
+ #maxMemSize will the whole available GPU memory.
1054
+ gpu:
1055
+ initMemSize: 2048 # Gpu Memory Pool init size
1056
+ maxMemSize: 4096 # Gpu Memory Pool Max size
1057
+
1058
+ # Any configuration related to the streaming node server.
1059
+ streamingNode:
1060
+ ip: # TCP/IP address of streamingNode. If not specified, use the first unicastable address
1061
+ port: 22222 # TCP port of streamingNode
1062
+ grpc:
1063
+ serverMaxSendSize: 268435456 # The maximum size of each RPC request that the streamingNode can send, unit: byte
1064
+ serverMaxRecvSize: 268435456 # The maximum size of each RPC request that the streamingNode can receive, unit: byte
1065
+ clientMaxSendSize: 268435456 # The maximum size of each RPC request that the clients on streamingNode can send, unit: byte
1066
+ clientMaxRecvSize: 268435456 # The maximum size of each RPC request that the clients on streamingNode can receive, unit: byte
1067
+
1068
+ # Any configuration related to the streaming service.
1069
+ streaming:
1070
+ walBalancer:
1071
+ # The interval of balance task trigger at background, 1 min by default.
1072
+ # It's ok to set it into duration string, such as 30s or 1m30s, see time.ParseDuration
1073
+ triggerInterval: 1m
1074
+ # The initial interval of balance task trigger backoff, 50 ms by default.
1075
+ # It's ok to set it into duration string, such as 30s or 1m30s, see time.ParseDuration
1076
+ backoffInitialInterval: 50ms
1077
+ backoffMultiplier: 2 # The multiplier of balance task trigger backoff, 2 by default
1078
+ txn:
1079
+ defaultKeepaliveTimeout: 10s # The default keepalive timeout for wal txn, 10s by default
1080
+
1081
+ # Any configuration related to the knowhere vector search engine
1082
+ knowhere:
1083
+ enable: true # When enable this configuration, the index parameters defined following will be automatically populated as index parameters, without requiring user input.
1084
+ DISKANN:
1085
+ build:
1086
+ max_degree: 56 # Maximum degree of the Vamana graph
1087
+ pq_code_budget_gb_ratio: 0.125 # Size limit on the PQ code (compared with raw data)
1088
+ search_cache_budget_gb_ratio: 0.1 # Ratio of cached node numbers to raw data
1089
+ search_list_size: 100 # Size of the candidate list during building graph
1090
+ search:
1091
+ beam_width_ratio: 4 # Ratio between the maximum number of IO requests per search iteration and CPU number