#PARTITION TABLES ON DBSCHEMA CODE#

The following example demonstrates reading the content of a Vertica table named test into a DataFrame.

The number of Spark partitions for the load from the Vertica job, each partition creates a JDBC connection. You can provide a specific IP address or resolvable name such as. This value is used to make the initial connection to Vertica and look up all the other Vertica node IPs. The schema defaults to РІР‚СљpublicРІР‚Сњ, but may be changed using the dbschema optional parameter. This user must have CREATE and INSERT privileges in the Vertica schema. Also, if the Vertica table is an external table, the underlying file(s) on which the external table is based must be accessible on all Vertica nodes. Note that the Vertica table must be segmented by hash or by an expression that returns non-negative integer values. The name of the target Vertica table or view to save your Spark DataFrame.

To connect to Vertica, you must specify the following parameters for the options in ('.DefaultSource').options(.).

#PARTITION TABLES ON DBSCHEMA CODE#

The following code sample illustrates how you can create an in-memory DataFrame by invoking SQLContext.read function, using VerticaРІР‚в„ўs .DefaultSource formatter. The Vertica Connector for Apache Spark data source API supports both parallel write and read operations. Loading Vertica Data into a Spark DataFrame Using the Vertica Data Source API

Partition tables on dbschema