Transcription

Role-Based AccessControl (RBAC) forKafka ConnectYeva Byzek, 2019 Confluent, Inc.

Table of ContentsIntroduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Target Audience.1Role-Based Access Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2Overview .Benefits.2.4Confluent Metadata Service.4Connect Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5Connect Worker .Connector.510Personas and Roles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Confluent CLI for Role Bindings .13.15.18.18Connect Cluster AdministratorConnector Submitter .ConnectorRole Binding Summary .22Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24Appendix. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25Cluster IDs .25

Role-Based Access Control (RBAC) for Kafka ConnectIntroductionRole-Based Access Control (RBAC) ensures that only authorized clients haveappropriate access to system resources. These resources include those available acrossservices in Confluent Platform: Kafka brokers Kafka Connect KSQL Confluent Schema Registry Confluent Control Center Confluent REST ProxyRBAC defines granular privileges for users and service accounts to different resources.We will review basic RBAC concepts and then dive into using RBAC specifically withKafka Connect and connectors.Target AudienceThe reader should understand basic principles of Apache Kafka and Kafka Connect,and understand how to deploy security across the services in Confluent Platform.Refer to the Connect and security documentation for prerequisite reading. 2014-2020 Confluent, Inc.1

Role-Based Access Control (RBAC) for Kafka ConnectRole-Based Access ControlOverviewBecause Kafka streams events that may contain extremely sensitive data, customersoften want to implement very strict rules that control who has access to this data andthe services in Confluent Platform.For basic, simple authorization, a user could define ACLs to allow or deny specific usersaccess to certain resources. However, these ACLs have tangible limitations: How do you efficiently manage privileges across an organization with hundreds ofusers? What if you wanted to allow a user to configure a sink connector that wouldconsume from a topic and send the messages to an end system? Without RBAC, a user with Confluent Control Center UI access is either asuperUser or a readOnly user with no middle ground for access—how do yourestrict a user to read from some topics but not others? What if you wanted to reduce your system dependencies on ZooKeeper, which isused to store the ACLs? How do you control granular access across all services in the Confluent Platform,including connectors, KSQL, REST Proxy, etc.? How do you match the identities of a business workflow that includes both users(humans) and service accounts (applications)?RBAC leverages predefined role assignments to determine who can access specificresources and what actions an individual user can perform within those resources. Anadministrator assigns predefined roles to users and groups; each user or group can beassigned multiple roles. Certain privileged users, such as the UserAdmin or SystemAdmin,assign roles to users and groups, and then map specific resources to those user roles. 2014-2020 Confluent, Inc.2

Role-Based Access Control (RBAC) for Kafka ConnectThe ResourceOwner role also has AlterAccess permissions on the resources to whichthey are bound, allowing them to delegate management of permissions to other users.Consequently, a ResourceOwner in a finance department can grant departmentmembers access to resources, perhaps to topics that use the prefix finance , forexample.User administrators can add LDAP users and groups, making it quicker and easier toconfigure authentication and authorization centrally for the various ConfluentPlatform resources used in an organization. With RBAC, the user administrator canmap roles to LDAP users and groups that are granted access to specific resources, viaa "role binding." These role bindings can be at the user level or group level. Group-levelbindings enable administrators to avoid having to grant explicit access to individualusers across every component. 2014-2020 Confluent, Inc.3

Role-Based Access Control (RBAC) for Kafka ConnectBenefitsRBAC benefits include: Robust framework that centralizes authentication and authorization in theConfluent Metadata Service (MDS) Consistent behavior across the Confluent Platform such that all services in theevent streaming platform can authorize users with the same mechanism KSQL supports impersonation for Interactive Queries so that it passes usercredentials transparently from the end user to the cluster REST Proxy supports impersonation so that it passes user credentialstransparently from the end user to the cluster Administrators can differentiate and authorize individual roles With a unified security CLI, administrators can define RBAC role bindings acrossthe entire Confluent PlatformConfluent Metadata ServiceConfluent Metadata Service (MDS) offers a single, centralized configuration contextthat binds and enforces a Kafka cluster configuration across different resources, suchas topics, connectors, and Schema Registry subjects. MDS acts as the central authorityfor all authorization, and it saves administrators from the complex and timeconsuming task of defining and assigning roles for each resource on an individual basis.It can be integrated with LDAP to provide authentication and refreshable bearertokens for impersonation. MDS is the master record for these role bindings, and allcomponents in the Confluent Platform communicate with MDS to ensure that after arole binding is set, users can’t gain access via another API or Confluent Control Centerto gain unauthorized access to resources. 2014-2020 Confluent, Inc.4

Role-Based Access Control (RBAC) for Kafka ConnectConnect ConfigurationBefore the introduction of RBAC, any user that could authenticate with Kafka Connectcould take any action on the connectors or Kafka topic data. However, with RBAC,Connect administrators can grant granular access to users and service accounts, withconnector-based authorization and role-based access. They can create multi-tenantConnect clusters that are shared between many departments in an enterprise. Sharinga Connect cluster and scaling it is especially compelling with the improvements madein Confluent Platform 5.3 that enable 10s to 100s of connectors per Kafka Connectcluster.The rest of this paper describes the workflow for enabling RBAC on Connect. Beforeproceeding to the sections below, ensure that your Kafka cluster is properly configuredfor RBAC, and refer to the RBAC documentation as needed.Connect WorkerThe Connect cluster administrator will need to configure all the Kafka Connectworkers and start the Connect cluster. There are many configuration parameters thatcan be set, and this paper focuses on a subset of those required for RBAC.For consistency with the RBAC demo that uses Hash Login Service with users, theexamples here do not have the configuration required to integrate with LDAP. If youneed more information on required LDAP configuration, refer to the LDAP Authorizerdocumentation.After configuring the Connect worker, create the appropriate role bindings describedin the section Personas and Roles. 2014-2020 Confluent, Inc.5

Role-Based Access Control (RBAC) for Kafka ConnectRefer to an example of the delta Connect configuration required to be added to your existing Connect configuration file. This configuration ispart of a demo all running on a localhost (e.g., bootstrap server atlocalhost:9092 and MDS at localhost:8090), so you’ll need to adapt itto your specific environment. 2014-2020 Confluent, Inc.6

Role-Based Access Control (RBAC) for Kafka Connect RBAC authentication and authorization: enable communication between theConnect worker and the Kafka cluster, basic token authentication between theConnect worker and MDS, and authentication for the Connect REST API 2014-2020 Confluent, Inc.7

Role-Based Access Control (RBAC) for Kafka Connect# Configuration required for communication with the Kafka clusterbootstrap.servers localhost:9092security.protocol SASL PLAINTEXTsasl.mechanism OAUTHBEARERsasl.login.callback.handler.class nUserLoginCallbackHandlersasl.jaas.config BearerLoginModule required username "connect" password "connect1"metadataServerUrls "http://localhost:8090";# Enables basic and bearer authentication for requests made to theworkerrest.servlet.initializor.classes tallBearerOrBasicSecurityHandler# The path to a directory containing public keys that should be usedto verify json web tokens during authenticationpublic.key.path /tmp/tokenPublicKey.pem# The location of a running metadata service; used to verify thatrequests are authorized by the users that make themconfluent.metadata.bootstrap.server.urls http://localhost:8090# Credentials to use when communicating with the MDS; these shouldusually match the ones used for communicating with Kafkaconfluent.metadata.basic.auth.user.info ntials.provider BASIC## REST extensions: RBAC and Secret Registry ### Installs the RBAC and Secret Registry REST extensionsrest.extension.classes cretRegistryExtension 2014-2020 Confluent, Inc.8

Role-Based Access Control (RBAC) for Kafka Connect Clients embedded within each Connect worker: producer, consumer, andAdminClientproducer.security.protocol SASL PLAINTEXTproducer.sasl.mechanism ass nUserLoginCallbackHandler# Intentionally omitting producer.sasl.jaas.config to forceconnectors to use their ownconsumer.security.protocol SASL PLAINTEXTconsumer.sasl.mechanism ass nUserLoginCallbackHandler# Intentionally omitting consumer.sasl.jaas.config to forceconnectors to use their ownadmin.security.protocol SASL PLAINTEXTadmin.sasl.mechanism OAUTHBEARERadmin.sasl.login.callback.handler.class nUserLoginCallbackHandler# Intentionally omitting admin.sasl.jaas.config to force connectorsto use their own# Allow producer/consumer/admin client overrides (this enables perconnector principals)connector.client.config.override.policy All 2014-2020 Confluent, Inc.9

Role-Based Access Control (RBAC) for Kafka Connect Connect Secret Registry: enable Connect to store encrypted Connect credentialsin a topic exposed through a REST APIconfig.providers secretconfig.providers.secret.class s.secret.param.master.encryption.key re.topic otstrap.servers tore.security.protocol SASL sasl.mechanism e.sasl.login.callback.handler.class aram.kafkastore.sasl.jaas.config BearerLoginModule requiredusername "connect" password "connect1"metadataServerUrls "http://localhost:8090";ConnectorAdditional connector configuration includes: RBAC authentication and authorization: enable basic authentication between theconnector and Kafka Refer to an example of the source connector or sink connector. 2014-2020 Confluent, Inc.10

Role-Based Access Control (RBAC) for Kafka Connect# Source connectorproducer.override.sasl.jaas.config BearerLoginModule required username "connector"password "connector1" metadataServerUrls "http://localhost:8090";# Sink connectorconsumer.override.sasl.jaas.config BearerLoginModule required username "connector"password "connector1" metadataServerUrls "http://localhost:8090";This configuration is part of a demo all running on a local host, so you’ll need to adaptit to your specific environment.The connector should be submitted after Kafka Connect has successfully started andwith the appropriate role bindings described in the section Personas and Roles. 2014-2020 Confluent, Inc.11

Role-Based Access Control (RBAC) for Kaf