• ICT Government Architects Must know and understand:

    As architects , many of us have grown from highly technical positions

    where our success was derived mainly from our ability to talk to machines.

    However, in the role of architect much of our communication is now done with

    our fellow human beings. Whether it’s talking to developers about the benefits

    of employing a specific pattern, or explaining to management the cost-benefit

    tradeoffs of buying middleware, communication is core to our success.

    Architects Must know and undestand:

  • Secularity

    Secularism.... One form of secularism is asserting the right to be free from religious rule and teachings, or, in a state declared to be neutral on matters of belief, from the imposition by government of religion or religious practices upon its people. The reason that this is an important argument is because the United States is a secular society, which means that the social structure is not based on or tied to any one particular religion. In sociology, the process by which a society moves away from a religious framework or foundation is known as secularization.

  • A curated list of insanely awesome ML libraries

    A curated list of insanely awesome libraries, packages and resources for Quants (Quantitative Finance)

    Languages

    Python

    R

    Matlab

    Julia

    Java

    JavaScript

    Haskell

    Scala

    Ruby

    CSharp

    Frameworks - frameworks that support different languages

    Reproducing Works - repositories that reproduce books and papers results or implement examples

    Python

    Numerical Libraries & Data Structures

    numpy - NumPy is the fundamental package for scientific computing with Python.

    scipy - SciPy (pronounced “Sigh Pie”) is a Python-based ecosystem of open-source software for mathematics, science, and engineering.

    pandas - pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language.

    quantdsl - Domain specific language for quantitative analytics in finance and trading

    statistics - Builtin Python library for all basic statistical calculations

    sympy - SymPy is a Python library for symbolic mathematics.

    pymc3 - Probabilistic Programming in Python: Bayesian Modeling and Probabilistic Machine Learning with Theano

    Financial Instruments

    PyQL - QuantLib's Python port

    pyfin - Basic options pricing in Python

    vollib - vollib is a python library for calculating option prices, implied volatility and greeks.

    QuantPy - A framework for quantitative finance In python

    Finance-Python - Python tools for Finance

    ffn - A financial function library for Python

    pynance - PyNance is open-source software for retrieving, analysing and visualizing data from stock and derivatives markets.

    tia - Toolkit for integration and analysis

    hasura/base-python-dash - Hasura quickstart to deploy Dash framework. Written on top of Flask, Plotly.js, and React.js, Dash is ideal for building data visualization apps with highly custom user interfaces in pure Python

    hasura/base-python-bokeh - Hasura quickstart to visualize data with bokeh library

    Trading & Backtesting

    TA-Lib - perform technical analysis of financial market data

    trade - trade is a Python framework for the development of financial applications.

    zipline - Pythonic algorithmic trading library

    QuantSoftware Toolkit - Python-based open source software framework designed to support portfolio construction and management.

    quantitative - Quantitative finance, and backtesting library

    analyzer - Python framework for real-time financial and backtesting trading strategies

    bt - Flexible Backtesting for Python

    backtrader - Python Backtesting library for trading strategies

    pythalesians - Python library to backtest trading strategies, plot charts, seamlessly download market data, analyse market patterns etc.

    pybacktest - Vectorized backtesting framework in Python / pandas, designed to make your backtesting easier.

    pyalgotrade - Python Algorithmic Trading Library

    tradingWithPython - A collection of functions and classes for Quantitative trading

    pandas_talib - A Python Pandas implementation of technical analysis indicators

    algobroker - This is an execution engine for algo trading

    pysentosa - Python API for sentosa trading system

    finmarketpy - Python library for backtesting trading strategies and analyzing financial markets

    binary-martingale - Computer program to automatically trade binary options martingale style

    Risk Analysis

    pyfolio - Portfolio and risk analytics in Python

    qrisk - Common financial risk and performance metrics

    fecon235 - Computational tools for financial economics include: Gaussian Mixture model of leptokurtotic risk, adaptive Boltzmann portfolios.

    finance - Financial Risk Calculations. Optimized for ease of use through class construction and operator overload.

    qfrm - Quantitative Financial Risk Management: awesome OOP tools for measuring, managing and visualizing risk of financial instruments and portfolios.

    visualize-wealth - Portfolio construction and quantitative analysis

    VisualPortfolio - This tool is used to visualize the perfomance of a portfolio

    Time Series

    ARCH - ARCH models in Python

    statsmodels - Python module that allows users to explore data, estimate statistical models, and perform statistical tests.

    dynts - Python package for timeseries analysis and manipulation

    PyFlux - Python library for timeseries modelling and inference (frequentist and Bayesian) on models

    tsfresh - Automatic extraction of relevant features from time series

    hasura/quandl-metabase - Hasura quickstart to visualize Quandl's timeseries datasets with Metabase

    Calendars

    tradingcalendar - Stock Exchange Trading Calendar

    bizdays - Business days calculations and utilities

    pandas_market_calendars - Exchange calendars to use with pandas for trading applications

    Data Sources

    findatapy - Python library to download market data via Bloomberg, Quandl, Yahoo etc.

    googlefinance - Python module to get real-time stock data from Google Finance API

    yahoo-finance - Python module to get stock data from Yahoo! Finance

    pandas-datareader - Python module to get data from various sources (Google Finance, Yahoo Finance, FRED, OECD, Fama/French, World Bank, Eurostat...) into Pandas datastructures such as DataFrame, Panel with a caching mechanism

    pandas-finance - High level API for access to and analysis of financial data

    pyhoofinance - Rapidly queries Yahoo Finance for multiple tickers and returns typed data for analysis

    yfinanceapi - Finance API for Python

    yql-finance - yql-finance is simple and fast https://developer.yahoo.com/yql/console/ python API. API returns stock closing prices for current period of time and current stock ticker (i.e. APPL, GOOGL).

    ystockquote - Retrieve stock quote data from Yahoo Finance

    wallstreet - Real time stock and option data

    stock_extractor - General Purpose Stock Extractors from Online Resources

    Stockex - Python wrapper for Yahoo! Finance API

    finsymbols - Obtains stock symbols and relating information for SP500, AMEX, NYSE, and NASDAQ

    FRB - Python Client for FRED® API

    inquisitor - Python Interface to Inquirim.com API

    yfi - Yahoo! YQL library

    chinesestockapi - Python API to get Chinese stock price

    exchange - Get current exchange rate

    ticks - Simple command line tool to get stock ticker data

    pybbg - Python interface to Bloomberg COM APIs

    ccy - Python module for currencies

    tushare - A utility for crawling historical and Real-time Quotes data of China stocks

    jsm - Get the japanese stock market data

    cn_stock_src - Utility for retrieving basic China stock data from different sources

    coinmarketcap - Python API for coinmarketcap

    after-hours - Obtain pre market and after hours stock prices for a given symbol

    bronto-python - Bronto API Integration for Python

    pytdx - Python Interface for retrieving chinese stock realtime quote data from TongDaXin Nodes

    pdblp - A simple interface to integrate pandas and the Bloomberg Open API

    tiingo - Python interface for daily composite prices/OHLC/Volume + Real-time News Feeds, powered by the Tiingo Data Platform.

    IEX - Python Interface for retrieving real-time and historical prices and equities data from The Investor's Exchange.

    Excel Integration

    xlwings - Make Excel fly with Python!

    openpyxl - Read/Write Excel 2007 xlsx/xlsm files

    xlrd - Library for developers to extract data from Microsoft Excel spreadsheet files

    xlsxwriter - Write files in the Excel 2007+ XLSX file format

    xlwt - Library to create spreadsheet files compatible with MS Excel 97/2000/XP/2003 XLS files, on any platform.

    DataNitro - DataNitro also offers full-featured Python-Excel integration, including UDFs. Trial downloads are available, but users must purchase a license.

    xlloop - XLLoop is an open source framework for implementing Excel user-defined functions (UDFs) on a centralised server (a function server).

    expy - The ExPy add-in allows easy use of Python directly from within an Microsoft Excel spreadsheet, both to execute arbitrary code and to define new Excel functions.

    pyxll - PyXLL is an Excel add-in that enables you to extend Excel using nothing but Python code.

    R

    Numerical Libraries & Data Structures

    xts - eXtensible Time Series: Provide for uniform handling of R's different time-based data classes by extending zoo, maximizing native format information preservation and allowing for user level customization and extension, while simplifying cross-class interoperability.

    data.table - Extension of data.frame: Fast aggregation of large data (e.g. 100GB in RAM), fast ordered joins, fast add/modify/delete of columns by group using no copies at all, list columns and a fast file reader (fread). Offers a natural and flexible syntax, for faster development.

    TSdbi - Provides a common interface to time series databases.

    tseries - Time Series Analysis and Computational Finance.

    its - Irregular time series.

    zoo - S3 Infrastructure for Regular and Irregular Time Series (Z's Ordered Observations).

    tis - Functions and S3 classes for time indexes and time indexed series, which are compatible with FAME frequencies.

    tfplot - Utilities for simple manipulation and quick plotting of time series data.

    tframe - A kernel of functions for programming time series methods in a way that is relatively independently of the representation of time.

    Data Sources

    IBrokers - Provides native R access to Interactive Brokers Trader Workstation API.

    Rblpapi - An R Interface to 'Bloomberg' is provided via the 'Blp API'.

    Quandl - Get Financial Data Directly Into R.

    Rbitcoin - Unified markets API interface (bitstamp, kraken, btce, bitmarket).

    GetTDData - Downloads and aggregates data for Brazilian government issued bonds directly from the website of Tesouro Direto.

    GetHFData - Downloads and aggregates high frequency trading data for Brazilian instruments directly from Bovespa ftp site.

    Financial Instruments

    RQuantLib - RQuantLib connects GNU R with QuantLib.

    quantmod - Quantitative Financial Modelling Framework

    Rmetrics - The premier open source software solution for teaching and training quantitative finance

    fAsianOptions - EBM and Asian Option Valuation

    fAssets - Analysing and Modelling Financial Assets

    fBasics - Markets and Basic Statistics

    fBonds - Bonds and Interest Rate Models

    fExoticOptions - Exotic Option Valuation

    fOptions - Pricing and Evaluating Basic Options

    fPortfolio - Portfolio Selection and Optimization

    portfolio - Analysing equity portfolios

    portfolioSim - Framework for simulating equity portfolio strategies

    stockPortfolio - Build stock models and analyze stock portfolios

    financial - Time value of money, cash flows and other financial functions.

    sde - Simulation and Inference for Stochastic Differential Equations

    termstrc - Zero-coupon Yield Curve Estimation

    YieldCurve - Modelling and estimation of the yield curve

    SmithWilsonYieldCurve - Constructs a yield curve by the Smith-Wilson method from a table of LIBOR and SWAP rates

    ycinterextra - Yield curve or zero-coupon prices interpolation and extrapolation

    opefimor - Option Pricing and Estimation of Financial Models in R

    maRketSim - Market simulator for R

    AmericanCallOpt - This package includes pricing function for selected American call options with underlying assets that generate payouts

    VarSwapPrice - Pricing a variance swap on an equity index

    RND - Risk Neutral Density Extraction Package

    LSMonteCarlo - American options pricing with Least Squares Monte Carlo method

    OptHedging - Estimation of value and hedging strategy of call and put options

    tvm - Time Value of Money Functions

    OptionPricing - Option Pricing with Efficient Simulation Algorithms

    credule - Credit Default Swap Functions

    derivmkts - Functions and R Code to Accompany Derivatives Markets

    FinCal - Package for time value of money calculation, time series analysis and computational finance

    r-quant - R code for quantitative analysis in finance

    binary_options - predicting stock direction for binary option trading

    options.studies - options trading studies functions for use with options.data package and shiny

    Trading

    TA-Lib - perform technical analysis of financial market data

    backtest - Exploring Portfolio-Based Conjectures About Financial Instruments

    pa - Performance Attribution for Equity Portfolios

    TTR - Technical Trading Rules

    QuantTools - Enhanced Quantitative Trading Modelling

    Risk Analysis

    PerformanceAnalytics - Econometric tools for performance and risk analysis

    Time Series

    tseries - Time Series Analysis and Computational Finance

    zoo - S3 Infrastructure for Regular and Irregular Time Series (Z's Ordered Observations)

    xts - eXtensible Time Series

    fGarch - Rmetrics - Autoregressive Conditional Heteroskedastic Modelling

    timeSeries - Rmetrics - Financial Time Series Objects

    rugarch - Univariate GARCH Models

    rmgarch - Multivariate GARCH Models

    Calendars

    RQuantLib

    timeDate - Chronological and Calendar Objects

    bizdays - Business days calculations and utilities

    Matlab

    FrameWorks

    QUANTAXIS - Integrated Quantitative Toolbox with Matlab

    Julia

    QuantLib.jl - Quantlib implementation in pure Julia.

    FinancialMarkets.jl - Describe and model financial markets objects using Julia

    Ito.jl - A Julia package for quantitative finance

    TALib.jl - A Julia wrapper for TA-Lib

    Miletus.jl - A financial contract definition, modeling language, and valuation framework

    Temporal.jl - Flexible and efficient time series class & methods

    Indicators.jl - Financial market technical analysis & indicators on top of Temporal

    Strategems.jl - Quantitative systematic trading strategy development and backtesting

    TimeSeries.jl - Time series toolkit for Julia

    MarketTechnicals.jl - Technical analysis of financial time series on top of TimeSeries

    MarketData.jl - Time series market data

    TimeFrames.jl - A Julia library that defines TimeFrame (essentially for resampling TimeSeries)

    Java

    JQuantLib - JQuantLib is a free, open-source, comprehensive framework for quantitative finance, written in 100% Java.

    finmat.net - Java library with algorithms and methodologies related to mathematical finance.

    quantcomponents - Free Java components for Quantitative Finance and Algorithmic Trading

    DRIP - Fixed Income, Asset Allocation, Transaction Cost Analysis, XVA Metrics Libraries.

    JavaScript

    Data Visualization

    QUANTAXIS_Visualziation an awesome visualization center based on quantaxis

    Haskell

    quantfin - quant finance in pure haskell

    hqfl - Haskell Quantitative Finance Library

    Scala

    QuantScale - Scala Quantitative Finance Library

    Scala Quant Scala library for working with stock data from IFTTT recipes or Google Finance.

    Ruby

    Jiji - Open Source Forex algorithmic trading framework using OANDA REST API.

    Frameworks

    QuantLib - The QuantLib project is aimed at providing a comprehensive software framework for quantitative finance.

    JQuantLib - Java port

    RQuantLib - R port

    QuantLibAddin - Excel support

    QuantLibXL - Excel support

    QLNet - .Net port

    PyQL - Python port

    QuantLib.jl - Julia port

    TA-Lib - perform technical analysis of financial market data

    CSharp

    QuantConnect - Lean Engine is an open-source fully managed C# algorithmic trading engine built for desktop and cloud usage.

    Reproducing Works

    Derman Papers - Notebooks that replicate original quantitative finance papers from Emanuel Derman.

    volatility-trading - A complete set of volatility estimators based on Euan Sinclair's Volatility Trading.

    quant - Quantitative Finance and Algorithmic Trading exhaust; mostly ipython notebooks based on Quantopian, Zipline, or Pandas.

  • The Basics: Security questions to get you started

  • Complex Systems

    The content of this page my change over time, depending on how much time I can dedicate to build it.

    A complex system is a system composed of interconnected parts that as a whole exhibit one or more properties (behavior among the possible properties) not obvious from the properties of the individual parts.
    In the last years, my investigation of complex systems can be summarized by the following (non-exhaustive, unordered, incomplete, nonlinear) random walking in the corresponding complex network of knodges:

    Complex systems theory

    But if you want a more complicated idea in a more simplified shape, you can take a look at this figure.

    Complex Networks

    A very nice application of network theory to real-world can be found in this 1st TED talk and in this 2nd TED talk given by Nicholas Christakis.

    Or you can take a look at the following video about the "Controllability of Complex Networks" (see the Barabasi Lab page dedicated to this project), which of course will capture your attention on complex networks:

    Nonlinear Dynamics and Time Series Analysis 

    1. Free Time Series - Free time series for linear and nonlinear time series analysis generated with TSUT
    2. Dynamical Models wiki - Examples of dynamical model simulated with RooTiSA.
    3. Complexity Animation - Nonlinearity and Complexity animations from BOC
    4. Software - Recommended software for Nonlinear Dynamics Analysis
    5. Links - Recommended links

    Complexity Animations

    • Diffusion Limited Aggregation [View]
    • Sea Things [View]
    • Lorenz Attractor [View]
    • Fractal Invader [View]
    • Simple Binary Network [View]
    • Binary Network Sound Machine [View]
    • Wolfram 8-bit 1D Cellular Automata [View]
    • One Legged Creatures [View]
    • Spring / Mass / Motor system [View]

    Software

     

    Netlib (a collection of mathematical software, papers, and databases) [Homepage]:

    • BLAS (Basic Linear Algebra Subprograms) [Homepage]
    • LAPACK (Linear Algebra PACKage) [Homepage]
    • ARMADILLO (C++ linear algebra library) [Homepage]

    Libraries:

    • FFTW (Fastest Fourier Transform in the West) [Homepage]
    • ATLAS (Automatically Tuned Linear Algebra Software) [Homepage]
    • IMKL (Intel Math Kernel Library) [Homepage]
    • ACML (AMD Core Math Library) [Homepage]
    • CEPHES (Mathematical Library) [Homepage]

    Klick to see chaos Funny video!

    Links

     

  • Short tip for a long thinker

  • e -Government: lack of communication and standards.

  • Artificial Intelligence

  • A list of artificial intelligence tools — for personal use (1/3)

    Artificial Intelligence and the fourth industrial revolution has made some considerable progress over the last couple of years. Most of this current progress that is usable has been developed for industry and business purposes, as you’ll see in coming posts. Research institutes and dedicated, specialized companies are working toward the ultimate goal of AI (cracking artificial general intelligence), developing open platforms and the looking into the ethics that follow suit. There are also a good handful of companies working on AI products for consumers, which is what we’ll be kicking this series of posts off with.

    Artificial intelligence is like climbing a tree to try and reach the moon; one can report steady progress, all the way to the top of the tree.

    As a note: a few of the products haven’t launched yet, and may be still in Beta, though are exciting ideas, well backed or look promising. Most you can use now.

    Let me know your feedback (including any edits, adds and removals)

    I’ve highlighted some of my favourites, enjoy —!

    Supported (and made intelligent) by:

  • Commonly used Machine Learning Algorithms (with Python and R) Codes)

    SUNIL RAY,
    Overview
    • Major focus on commonly used machine learning algorithms
    • Algorithms covered- Linear regression, logistic regression, Naive Bayes, kNN, Random forest, etc.
    • Learn both theory and implementation of these algorithms in R and python

    Introduction

    Google’s self-driving cars and robots get a lot of press, but the company’s real future is in machine learning, the technology that enables computers to get smarter and more personal.

    – Eric Schmidt (Google Chairman)

    We are probably living in the most defining period of human history. The period when computing moved from large mainframes to PCs to cloud. But what makes it defining is not what has happened, but what is coming our way in years to come.

© 2020