What is already supported
- View, create and delete laptops
- Help Writing Scala Code
- Auto completion
- Refactoring Rename
- Extract of a variable, method or parameter
- Go to declaration
- Usage Search
- Create and delete paragraphs
- Running paragraphs
- View paragraph output
- Basic rendering support
So far, help with writing code only works for Scala. Other languages will be supported later. In addition, we are not going to limit ourselves to Zeppelin support and plan to add more features that will be useful for those who work with big data.
If you can't wait to see Big Data Tools in action, check out this short video:
Better try the plugin yourself! How to do this, read on.
How to use the plugin
- Make sure you are using IntelliJ IDEA Ultimate 2019.2 (note that the plugin does not work with EAP 2019.3 yet).
- Check if BashSupport, Python, and Scala plugins are installed (recent versions will work). So far, they will have to be installed manually - we will fix it soon.
- Install the latest Big Data Tools plugin build.
- If everything works as it should, after restarting the IDE, the Big Data Tools window will appear on the right. Open it, click on the “+” icon on the toolbar in the window and select Zeppelin.
- In the Zeppelin Connection dialog box, specify the settings for connecting your Zeppelin instance (host, port, credentials, etc.). To test the connection, use Test Connection.
- After configuring the Zeppelin configuration, you will see this instance in the tool window along with the laptop tree. Now you can manage laptops and open them in the editor.
What's next?
First of all, we are not going to limit ourselves to integration with Zeppelin and Spark, which we, of course, will improve. We plan to support more tools for working with big data: add integration with distributed file systems such as HDFS and S3, implement the integrated Parquet file viewer and improve SQL support.
Here is an example plan for our work in the foreseeable future:
- November 2019: IntelliJ IDEA Ultimate 2019.3 EAP compatible.
- November 2019: integration with S3 (explorer).
- December 2019: basic SQL support (highlighting, auto-completion, auto-formatting, etc.).
- January 2020: Integration with HDFS (Explorer) and Parquet viewer.
- February 2020: basic integration with Hadoop and Spark (user interfaces YARN, Spark, etc.).
- Spring / Summer 2020: Python support in Zeppelin laptops (we also consider the compatibility of the plugin with PyCharm Professional Edition, we will inform about this later).
This is an inconclusive plan. We will adjust it based on your feedback to make sure that we are focused on the right things. Please tell us about errors and share your opinion about what features are missing and how, in your opinion, this or that function should work.
What else is important to know about the new plugin
The Big Data Tools plugin is only compatible with IntelliJ IDEA Ultimate and is not available in Community Edition. We do not yet know if he will be in Community Edition. At the moment, we do not have such plans.
The same goes for other development environments. We do not yet know whether the plugin will be compatible with other IDEs. At the moment, we only have a preliminary plan to add compatibility with PyCharm Professional Edition, as soon as Python support is added to the plugin. As the plug-in develops, we will be able to better understand where we should move in the context of its support in other IDEs.
I do not have a license for IntelliJ IDEA Ultimate. How can I try the plugin?
- If you are actively involved in any open source project related to big data, write to us.
- If you want to write about Big Data Tools, contact us and we will provide you with an extended trial version so you can evaluate the plugin.
- If you are neither a committer nor a blogger, but are interested in big data, contact us for an extended trial version (for example, up to 3 months). This offer is limited.
How to leave a review
We are really looking forward to your feedback on the plugin. You can ask any questions in the comments to this post. Leave comments and suggestions in our tracker .
You can also share your impressions about the plugin by going through a short survey . It only takes a minute.
Constructive criticism will help us make your work with Spark, Zeppelin and Big Data as convenient and efficient as possible. Therefore, do not hesitate to share your feedback and suggestions - this is important to us.
PS It is especially nice that the plug-in coincidence coincided with our participation in Spark AI Summit. So if you suddenly find yourself at this conference, be sure to come to our booth: you can look at the plugin in action and share your impressions.