All Projects โ†’ RandomFractals โ†’ Vscode Data Preview

RandomFractals / Vscode Data Preview

Licence: apache-2.0
Data Preview ๐Ÿˆธ extension for importing ๐Ÿ“ค viewing ๐Ÿ”Ž slicing ๐Ÿ”ช dicing ๐ŸŽฒ charting ๐Ÿ“Š & exporting ๐Ÿ“ฅ large JSON array/config, YAML, Apache Arrow, Avro, Parquet & Excel data files

Programming Languages

typescript
32286 projects

Projects that are alternatives of or similar to Vscode Data Preview

Choetl
ETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)
Stars: โœญ 372 (+51.84%)
Mutual labels:  json, csv, yaml, avro, parquet
Just Dashboard
๐Ÿ“Š ๐Ÿ“‹ Dashboards using YAML or JSON files
Stars: โœญ 1,511 (+516.73%)
Mutual labels:  json, csv, yaml, data
Sheetjs
๐Ÿ“— SheetJS Community Edition -- Spreadsheet Data Toolkit
Stars: โœญ 28,479 (+11524.08%)
Mutual labels:  excel, json, csv, data
Rumble
โ›ˆ๏ธ Rumble 1.11.0 "Banyan Tree"๐ŸŒณ for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more
Stars: โœญ 58 (-76.33%)
Mutual labels:  json, csv, avro, parquet
Iso 3166 Countries With Regional Codes
ISO 3166-1 country lists merged with their UN Geoscheme regional codes in ready-to-use JSON, XML, CSV data sets
Stars: โœญ 1,372 (+460%)
Mutual labels:  json, csv, data
Schemer
Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.
Stars: โœญ 97 (-60.41%)
Mutual labels:  json, avro, parquet
Config
๐Ÿ“ Go config manage(load,get,set). support JSON, YAML, TOML, INI, HCL, ENV and Flags. Multi file load, data override merge, parse ENV var. Goๅบ”็”จ้…็ฝฎๅŠ ่ฝฝ็ฎก็†๏ผŒๆ”ฏๆŒๅคš็งๆ ผๅผ๏ผŒๅคšๆ–‡ไปถๅŠ ่ฝฝ๏ผŒ่ฟœ็จ‹ๆ–‡ไปถๅŠ ่ฝฝ๏ผŒๆ”ฏๆŒๆ•ฐๆฎๅˆๅนถ๏ผŒ่งฃๆž็Žฏๅขƒๅ˜้‡ๅ
Stars: โœญ 225 (-8.16%)
Mutual labels:  config, json, yaml
Data Store
Easily get, set and persist config data. Fast. Supports dot-notation in keys. No dependencies.
Stars: โœญ 120 (-51.02%)
Mutual labels:  config, json, data
Vscode Vega Viewer
VSCode extension for Interactive Preview of Vega & Vega-Lite maps ๐Ÿ—บ๏ธ & graphs ๐Ÿ“ˆ
Stars: โœญ 75 (-69.39%)
Mutual labels:  vscode, extension, viewer
Geo Data Viewer
๐Ÿ—บ๏ธ Geo Data Viewer w/0 Py ๐Ÿ || pyWidgets โš™๏ธ || pandas ๐Ÿผ || @reactjs โš›๏ธ required to gen. some snazzy maps ๐Ÿ—บ๏ธ with keplerGL ...
Stars: โœญ 115 (-53.06%)
Mutual labels:  data, vscode, viewer
Konf
A type-safe cascading configuration library for Kotlin/Java/Android, supporting most configuration formats
Stars: โœญ 225 (-8.16%)
Mutual labels:  config, json, yaml
Filecontextcore
FileContextCore is a "Database"-Provider for Entity Framework Core and adds the ability to store information in files instead of being limited to databases.
Stars: โœญ 91 (-62.86%)
Mutual labels:  excel, json, csv
Tabtoy
้ซ˜ๆ€ง่ƒฝ่กจๆ ผๆ•ฐๆฎๅฏผๅ‡บๅ™จ
Stars: โœญ 1,302 (+431.43%)
Mutual labels:  excel, json, csv
Cfgdiff
diff(1) all your configs
Stars: โœญ 138 (-43.67%)
Mutual labels:  config, json, yaml
Dbwebapi
(Migrated from CodePlex) DbWebApi is a .Net library that implement an entirely generic Web API (RESTful) for HTTP clients to call database (Oracle & SQL Server) stored procedures or functions in a managed way out-of-the-box without any configuration or coding.
Stars: โœญ 84 (-65.71%)
Mutual labels:  excel, json, csv
Gray Matter
Contributing Pull requests and stars are always welcome. For bugs and feature requests, please create an issue.
Stars: โœญ 2,105 (+759.18%)
Mutual labels:  config, yaml, data
Rq
Record Query - A tool for doing record analysis and transformation
Stars: โœญ 1,808 (+637.96%)
Mutual labels:  json, yaml, avro
Uxdm
๐Ÿ”€ UXDM helps developers migrate data from one system or format to another.
Stars: โœญ 159 (-35.1%)
Mutual labels:  array, json, csv
Chronicle Wire
A Java Serialisation Library that supports multiple formats
Stars: โœญ 204 (-16.73%)
Mutual labels:  json, csv, yaml
Countries States Cities Database
๐ŸŒ World countries, states, regions, provinces, cities, towns in JSON, SQL, XML, PLIST, YAML, and CSV. All Countries, States, Cities with ISO2, ISO3, Country Code, Phone Code, Capital, Native Language, Timezones, Latitude, Longitude, Region, Subregion, Flag Emoji, and Currency. #countries #states #cities
Stars: โœญ 1,130 (+361.22%)
Mutual labels:  json, csv, yaml

vscode-data-preview

Build Status Apache-2.0 License https://ko-fi.com/dataPixy

Version Installs Downloads

Trending-Weekly Trending-Monthly

Data Preview ๐Ÿˆธ extension for importing ๐Ÿ“ค viewing ๐Ÿ”Ž slicing ๐Ÿ”ช dicing ๐ŸŽฒ charting ๐Ÿ“Š & exporting ๐Ÿ“ฅ large .json array .arrow .avro .parquet data files, .config .env .properties .ini .yml configurations files, .csv/.tsv & .xlsx/.xlsb Excel files and .md markdown tables with Perspective - streaming data analytics WebAssembly library.

Data Preview

๐Ÿˆธ Features

  • Preview .json .arrow .avro .parquet .yml .csv/.tsv & .xlsx/.xlsb data files in a Data Grid w/Sorting & Filtering
  • Grid Data Summary display w/Aggregate Functions, Row & Column Pivots (a.k.a. Group By & Split By)
  • Basic Charts ๐Ÿ“Š creation w/Aggregate Functions, Row & Column Pivots
  • Pluggable Charting ๐Ÿ“Š libraries for bult-in Charts: d3fc || highcharts
  • Persistent Data Preview Settings (View, Sort, Filter, Pivots, etc.) for restore of open Data View panels on VSCode Reload
  • Mulptiple Spreadsheets Data Preview for Excel data files
  • Markdown tables data preview for .md documentation files
  • Data .schema.json generation for Arrow & Avro Data Schema Text Previews in JSON format
  • Binary Data files .json generation for Arrow, Avro & Excel formats for Text Data Preview
  • Property Grid display for .json .config .env .properties .ini & .yml configuration files
  • Open Data Preview on Side option for slim data || config files
  • Quick Launch new Data Preview input box for data files in open workspace
  • Save Filtered Data Grid || Chart ๐Ÿ“Š Data in .arrow .csv .json(s) .yml & .properties formats
  • Save & Load Data View .config options
  • Dark, Light, Dense & High Contrast Blue Data Preview ๐Ÿˆธ Panel UI Themes

Next V.

  • Will include large text & binary data files loading & Apache Arrow data streaming.

Note: Data Preview ๐Ÿˆธ is already capable of loading a few 10+MB's large data files with 100+K records & extensive list of supported Data Formats you'll be hard pressed to find on VSCode marketplace in one extension.

See data/large/... data folder for sample large data files and Data View .configs you can try in Data Preview ๐Ÿˆธ.

Installation

Install Data Preview ๐Ÿˆธ via vscode Extensions tab (Ctrl+Shift+X) by searching for data preview || via VSCode marketplace search results.

List of Data Preview ๐Ÿˆธ extension config Settings, data.preview command(s), keyboard shortcut(s), augmented vscode UI context menus, added Data Language mappings, supported Data Files list & configurable Theme & Charts ๐Ÿ“Š Settings:

Data Preview Contributions

Configuration

Create User or Workspace Settings in vscode to change default Data Preview ๐Ÿˆธ extension Settings:

Setting Type Default Value Description
data.preview.theme string dark Data Preview UI Theme: dark, light, dense.light, dense.dark, or vaporwave (hight contrast blue theme)
data.preview.charts.plugin string d3fc Data Preview Charts ๐Ÿ“Š library to use for built-in charts: d3fc or highcharts
data.preview.create.json.files boolean false Creates .json data files for Arrow, Avro & Excel binary data formats
data.preview.create.json.schema boolean true Creates .schema.json files for Arrow & Avro metadata binary data formats
data.preview.openSavedFileEditor boolean true Opens created data file Content Editor on Data Save
data.preview.log.level string info Data Preview run log level: info or debug for issues troubleshooting

Data Preview ๐Ÿˆธ example using dark UI theme with d3fc Charts ๐Ÿ“Š Data View config, viewing superstore.arrow data file :)

Data Preview Dark

Usage Tips

Data Preview ๐Ÿˆธ Launch Tips

  1. Run View -> Command Palette...>Data: Preview Data command or Ctrl+Shift+D in an open .json .config .env .properties .ini .yml or .csv/.tsv text data file document to launch Data Preview panel.
  2. File -> Save (Ctrl+S) your text data file for immediate updates in an open Data Preview ๐Ÿˆธ panel.
  3. Right-click on a Binary .xlsx/.xlsb, .arrow or .avro data file in VSCode File Explorer to launch Data Preview panel.
  4. Use exposed explorer/context, editor/title or editor/title/context Preview Data ๐Ÿˆธ or Preview Data on Side context menu options to preview your data files.
  5. Click on the Data View Filename toolbar link to Load saved Data View Grid || Chart ๐Ÿ“Š .config.
  6. Click on the Data View ๐Ÿˆธ icon to Launch new Data Preview Panel for new view config changes.
  7. Use Open Data File or URL ๐Ÿ“ค option from Data View toolbar to launch new Data Preview ๐Ÿˆธ.
  8. Run View -> Command Palette...>Data: Preview Remote Data command or Ctrl+Shift+R to launch Data Preview for remote http(s) data files.

Data Preview Open Data File

Data Grid/Filter/Columns UX Tips

  1. Double click on the Grid Column header to Sort data by that column.
  2. Drag and drop a column from the left-side Columns control panel into Filter fields for data filtering (Group By, Split By, Sort, Filter).
  3. Drag columns up and down in the left-side Columns control panel to reorder displayed columns in the Data Grid.
  4. Uncheck a column in the Columns control panel to remove it from a Chart ๐Ÿ“Š or Data Grid display.

Usage Scenarios

Use Data Preview ๐Ÿˆธ to:

  • Load large data files for sorting, filtering & charting ๐Ÿ“Š
  • Export displayed data in a compact binary .arrow data format, .ods .xlsb || .xlsx spreadsheet format, or .csv .json .yml .md or .properties text formats
  • Preview .properites and other key-value pairs configuration files to reformat them or find set config option values
  • Generate Arrow & Avro .shema.json for metadata text preview of those binary data files
  • Generate .json files for text data preview of binary Excel files
  • Preview Excel files and workbooks with multiple workseets without opening Excel
  • Convert .csv or .tsv data to .json or .yml format
  • Extract, sort, filter and save markdown tables from .md documentation files
  • Use built-in Charts ๐Ÿ“Š for Exploratory Data Analysis

Supported JSON, Config, Binary & Excel Data File Formats

Tip: try sample data and Data View .config files from this repository data/... folders: data/arrow data/avro data/parquet data/config data/excel data/json data/yaml data/large

Vega datasets repository also has a broad collection of sample .csv & .json array data files you can try in Data Preview ๐Ÿˆธ

Data Preview ๐Ÿˆธ Files Matching Rules

{
  "when": "resourceFilename =~ /.*\\.(json|jsonl|json5|hjson|ndjson|arrow|arr|avro|parquet|env|config|properties|ini|yml|md|csv|tsv|txt|tab|dif|ods|xls|xlsb|xlsx|xlsm|xml|html)/",
  "command": "data.preview",
  "group": "navigation"
}

Data Preview ๐Ÿˆธ Files Loading Details

Note: .json .config & .yml configuration files that don't contain array data are converted to flat properties key/value pairs Object and displayed in a Property Grid Data View mode. See json.utils.ts for more info.

Data File Extension(s) File Type Data Parsing Library/Method Used Data Format Specification
.json .config .jsonl .ndjson text JSON.parse() https://json.org/
.json5 text json5/JSON5.parse() https://json5.org/
.hjson text hjson-js/Hjson.parse() https://hjson.org/
.arrow .arr binary apache-arrow/Table.from() https://arrow.apache.org/
.parquet binary parquets/ParquetReader.openFile() https://parquet.apache.org/documentation/latest
.properties .env text node-properties/properties.parse() https://en.wikipedia.org/wiki/.properties
.ini text node-properties/properties.parse() https://en.wikipedia.org/wiki/INI_file
.md text RegExp/markdownToCsv() https://en.wikipedia.org/wiki/Markdown
.yml .yaml text js-yaml/yaml.load() https://yaml.org/
.csv .tsv .txt .tab text perspective/perspectiveViewer.load(text) https://en.wikipedia.org/wiki/Comma-separated_values https://en.wikipedia.org/wiki/Tab-separated_values
.dif .ods .xls .xlsb .xlsx .xlsm .xml .html binary/text js-xlsx/XLSX.read() See https://github.com/SheetJS/js-xlsx#file-formats for more info on Excel file formats

See Data Manager API & src/data.providers folder for data loading and saving imlementation details.

Provided Chart ๐Ÿ“Š Types

  • Area Chart
  • Bar Chart
  • Candlesick Chart (d3fc only))
  • Heatmap
  • Line Chart
  • OHLC Chart (d3fc only)
  • Scatter Chart
  • Sunburst
  • Tree Map

Data Preview Chart Types

Supported Filter Functions

  • <, <=, ==, !=, >, >= for dates and number columns/fields
  • ==, !=, contains, in, not in, begins with, ends with for string fields and dictionaries
  • &, |, and, or, ==, != for bolean fields

Supported Aggregate Functions

  • any
  • avg
  • count
  • distinct count
  • dominant
  • first by index
  • last by index
  • last
  • high
  • low
  • mean
  • mean by count
  • median
  • pct sum parent
  • pct sum grand total
  • sum
  • sum abs
  • sum not null
  • unique

Recommended VSCode Extensions

Other extensions Data Preview ๐Ÿˆธ replaces, enhances or supplements for working with supported data file formats in VSCode:

Extension Description
Excel Viewer View Excel spreadsheets and CSV files
Avro Viewer .avro file viewer
avro-idl Avro IDL Syntax Highlighter
DotENV .env Syntax Highlighter
Ini for VSCode Provides outline view and section folding for INI files
Hjson Hjson language syntax support
JSON5 syntax Adds syntax highlighting of JSON5 files
NDJSON Colorizer Colorizes NDJSON (Newline Delimited JSON) files
YAML YAML Language Support by Red Hat, with built-in Kubernetes and Kedge syntax support
docs-yaml YAML schema validation and auto-completion for docs.microsoft.com authoring
YAML to JSON Convert YAML from clipboard or current document/selection to JSON and vice versa
Properties To Yaml Convert properties to yaml
Markdown Table Prettifier Transforms markdown tables to be more readable

Dev Log

See #DataPreview ๐Ÿˆธ tag on Twitter for the latest and greatest updates on this vscode extension and what's in store next.

Dev Build

$ git clone https://github.com/RandomFractals/vscode-data-preview
$ cd vscode-data-preview
$ npm install
$ code .

F5 to launch Data Preview extension VSCode debug session.

||

vscode-data-preview>vsce package

to generate VSIX Data Preview extension package from our latest for local dev install in VSCode.

Note: Use Help -> Toggle Developer Tools vscode menu option to view Data Preview console log.

Contributions

Any and all test, code or feedback contributions are welcome.

Open an issue or create a pull request to make this Data Preview ๐Ÿˆธ extension work better for all.

Backers

SheetJS Aman Mittal
support me on ko-fi.com
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].