Release v1.11.0

* potential fixes for #431
* go: fix formatting/gofmt/goreturns

.github/ISSUE_TEMPLATE.md

@@ -1,20 +1,36 @@
-Please answer the following questions. 
+<!-- This is a bug report template. By following the instructions below and
+filling out the sections with your information, you will help the us to get all
+the necessary data to fix your issue.
 
-### Which version of matterbridge are you using?
-run ```matterbridge -version```
+You can also preview your report before submitting it.
 
-### If you're having problems with mattermost please specify mattermost version. 
+Text between <!-- and --​> marks will be invisible in the report.
+-->
 
+<!-- If you have a configuration problem, please first try to create a basic configuration following the instructions on [the wiki](https://github.com/42wim/matterbridge/wiki/How-to-create-your-config) before filing an issue. -->
+
+
+### Environment
+<!-- run `matterbridge -version` -->
+<!-- If you're having problems with mattermost also specify the mattermost version. -->
+Version:
+
+<!-- What operating system are you using ? (be as specific as possible) -->
+Operating system:
+
+<!-- If you compiled matterbridge yourself:
+       * Specify the output of `go version` 
+       * Specify the output of `git rev-parse HEAD` -->
 
 ### Please describe the expected behavior.
 
 
 ### Please describe the actual behavior. 
-#### Use logs from running ```matterbridge -debug``` if possible.
+<!-- Use logs from running `matterbridge -debug` if possible. -->
 
 
 ### Any steps to reproduce the behavior?
 
 
 ### Please add your configuration file 
-#### (be sure to exclude or anonymize private data (tokens/passwords))
+<!-- (be sure to exclude or anonymize private data (tokens/passwords)) -->

.github/ISSUE_TEMPLATE/Bug_report.md

@@ -0,0 +1,26 @@
+---
+name: Bug report
+about: Create a report to help us improve. (Check the FAQ on the wiki first)
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots/debug logs**
+If applicable, add screenshots to help explain your problem.
+Use logs from running `matterbridge -debug` if possible.
+
+**Environment (please complete the following information):**
+ - OS: [e.g. linux]
+ - Matterbridge version: output of  `matterbridge -version`
+ - If self compiled: output of `git rev-parse HEAD`
+
+**Additional context**
+Please add your configuration file  (be sure to exclude or anonymize private data (tokens/passwords))

.github/ISSUE_TEMPLATE/Feature_request.md

@@ -0,0 +1,17 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.travis.yml

@@ -0,0 +1,51 @@
+language: go
+go:
+    #- 1.7.x
+    - 1.10.x
+      # - tip
+
+# we have everything vendored
+install: true
+
+env:
+    - GOOS=linux GOARCH=amd64
+      #    - GOOS=windows GOARCH=amd64
+      #- GOOS=linux GOARCH=arm
+
+matrix:
+  # It's ok if our code fails on unstable development versions of Go.
+  allow_failures:
+    - go: tip
+  # Don't wait for tip tests to finish. Mark the test run green if the
+  # tests pass on the stable versions of Go.
+  fast_finish: true
+
+notifications:
+      email: false
+
+before_script:
+  - MY_VERSION=$(git describe --tags)
+  - GO_FILES=$(find . -iname '*.go' | grep -v /vendor/)  # All the .go files, excluding vendor/
+  - PKGS=$(go list ./... | grep -v /vendor/)             # All the import paths, excluding vendor/
+#  - go get github.com/golang/lint/golint                 # Linter
+  - go get honnef.co/go/tools/cmd/megacheck              # Badass static analyzer/linter
+
+# Anything in before_script: that returns a nonzero exit code will
+# flunk the build and immediately stop. It's sorta like having
+# set -e enabled in bash. 
+script:
+ #- test -z $(gofmt -s -l $GO_FILES)  # Fail if a .go file hasn't been formatted with gofmt
+  - go test -v -race $PKGS            # Run all the tests with the race detector enabled
+ #  - go vet $PKGS                      # go vet is the official Go static analyzer
+  - megacheck $PKGS                   # "go vet on steroids" + linter
+  - /bin/bash ci/bintray.sh
+  #- golint -set_exit_status $PKGS     # one last linter
+
+deploy:
+  provider: bintray
+  edge:
+    branch: v1.8.47
+  file: ci/deploy.json
+  user: 42wim
+  key:
+     secure: "CeXXe6JOmt7HYR81MdWLua0ltQHhDdkIeRGBFbgd7hkb1wi8eF9DgpAcQrTso8NIlHNZmSAP46uhFgsRvkuezzX0ygalZ7DCJyAyn3sAMEh+UQSHV1WGThRehTtidqRGjetzsIGSwdrJOWil+XTfbO1Z8DGzfakhSuAZka8CM4BAoe3YeP9rYK8h+84x0GHfczvsLtXZ3mWLvQuwe4pK6+ItBCUg0ae7O7ZUpWHy0xQQkkWztY/6RAzXfaG7DuGjIw+20fhx3WOXRNpHCtZ6Bc3qERCpk0s1HhlQWlrN9wDaFTBWYwlvSnNgvxxMbNXJ6RrRJ0l0bA7FUswYwyroxhzrGLdzWDg8dHaQkypocngdalfhpsnoO9j3ApJhomUFJ3UoEq5nOGRUrKn8MPi+dP0zE4kNQ3e4VNa1ufNrvfpWolMg3xh8OXuhQdD5wIM5zFAbRJLqWSCVAjPq4DDPecmvXBOlIial7oa312lN5qnBnUjvAcxszZ+FUyDHT1Grxzna4tMwxY9obPzZUzm7359AOCCwIQFVB8GLqD2nwIstcXS0zGRz+fhviPipHuBa02q5bGUZwmkvrSNab0s8Jo7pCrel2Rz3nWPKaiCfq2WjbW1CLheSMkOQrjsdUd1hhbqNWFPUjJPInTc77NAKCfm5runv5uyowRLh4NNd0sI="

README-0.6.md

@@ -1,115 +0,0 @@
-# matterbridge
-
-Simple bridge between mattermost, IRC, XMPP, Gitter and Slack
-
-* Relays public channel messages between mattermost, IRC, XMPP, Gitter and Slack. Pick and mix.
-* Supports multiple channels.
-* Matterbridge can also work with private groups on your mattermost.
-
-Look at [matterbridge.conf.sample] (https://github.com/42wim/matterbridge/blob/master/matterbridge.conf.sample) for documentation and an example.
-
-## Changelog
-Since v0.6.1 support for XMPP, Gitter and Slack is added. More details in [changelog.md] (https://github.com/42wim/matterbridge/blob/master/changelog.md)
-
-## Requirements:
-Accounts to one of the supported bridges
-* [Mattermost] (https://github.com/mattermost/platform/)
-* [IRC] (http://www.mirc.com/servers.html)
-* [XMPP] (https://jabber.org)
-* [Gitter] (https://gitter.im)
-* [Slack] (https://www.slack.com)
-
-## binaries
-Binaries can be found [here] (https://github.com/42wim/matterbridge/releases/)
-* For use with mattermost 3.3.0+ [v0.6.1](https://github.com/42wim/matterircd/releases/tag/v0.6.1)
-* For use with mattermost 3.0.0-3.2.0 [v0.5.0](https://github.com/42wim/matterircd/releases/tag/v0.5.0)
-
-
-## Docker
-Create your matterbridge.conf file locally eg in ```/tmp/matterbridge.conf```
-
-```
-docker run -ti -v /tmp/matterbridge.conf:/matterbridge.conf 42wim/matterbridge:0.6.1
-```
-
-## Compatibility
-### Mattermost 
-* Matterbridge v0.6.1 works with mattermost 3.3.0 and higher [3.3.0 release](https://github.com/mattermost/platform/releases/tag/v3.3.0)
-* Matterbridge v0.5.0 works with mattermost 3.0.0 - 3.2.0 [3.2.0 release](https://github.com/mattermost/platform/releases/tag/v3.2.0)
-
-
-#### Webhooks version
-* Configured incoming/outgoing [webhooks](https://www.mattermost.org/webhooks/) on your mattermost instance.
-
-#### Plus (API) version
-* A dedicated user(bot) on your mattermost instance.
-
-
-## building
-Go 1.6+ is required. Make sure you have [Go](https://golang.org/doc/install) properly installed, including setting up your [GOPATH] (https://golang.org/doc/code.html#GOPATH)
-
-```
-cd $GOPATH
-go get github.com/42wim/matterbridge
-```
-
-You should now have matterbridge binary in the bin directory:
-
-```
-$ ls bin/
-matterbridge
-```
-
-## running
-1) Copy the matterbridge.conf.sample to matterbridge.conf in the same directory as the matterbridge binary.  
-2) Edit matterbridge.conf with the settings for your environment. See below for more config information.  
-3) Now you can run matterbridge. 
-
-```
-Usage of ./matterbridge:
-  -conf string
-        config file (default "matterbridge.conf")
-  -debug
-        enable debug
-  -plus
-        running using API instead of webhooks (deprecated, set Plus flag in [general] config)
-  -version
-        show version
-```
-
-## config
-### matterbridge
-matterbridge looks for matterbridge.conf in current directory. (use -conf to specify another file)
-
-Look at [matterbridge.conf.sample] (https://github.com/42wim/matterbridge/blob/master/matterbridge.conf.sample) for an example.
-
-### mattermost
-#### webhooks version
-You'll have to configure the incoming and outgoing webhooks. 
-
-* incoming webhooks
-Go to "account settings" - integrations - "incoming webhooks".  
-Choose a channel at "Add a new incoming webhook", this will create a webhook URL right below.  
-This URL should be set in the matterbridge.conf in the [mattermost] section (see above)  
-
-* outgoing webhooks
-Go to "account settings" - integrations - "outgoing webhooks".  
-Choose a channel (the same as the one from incoming webhooks) and fill in the address and port of the server matterbridge will run on.  
-
-e.g. http://192.168.1.1:9999 (192.168.1.1:9999 is the BindAddress specified in [mattermost] section of matterbridge.conf)
-
-#### plus version
-You'll have to create a new dedicated user on your mattermost instance.
-Specify the login and password in [mattermost] section of matterbridge.conf
-
-## FAQ
-Please look at [matterbridge.conf.sample] (https://github.com/42wim/matterbridge/blob/master/matterbridge.conf.sample) for more information first. 
-### Mattermost doesn't show the IRC nicks
-If you're running the webhooks version, this can be fixed by either:
-* enabling "override usernames". See [mattermost documentation](http://docs.mattermost.com/developer/webhooks-incoming.html#enabling-incoming-webhooks)
-* setting ```PrefixMessagesWithNick``` to ```true``` in ```mattermost``` section of your matterbridge.conf.
-
-If you're running the plus version you'll need to:
-* setting ```PrefixMessagesWithNick``` to ```true``` in ```mattermost``` section of your matterbridge.conf.
-
-Also look at the ```RemoteNickFormat``` setting.

README.md

@@ -1,17 +1,28 @@
 # matterbridge
-[![Gitter](https://img.shields.io/gitter/room/nwjs/nw.js.svg)](https://gitter.im/42wim/matterbridge) [![Join the IRC chat at https://webchat.freenode.net/?channels=matterbridgechat](https://img.shields.io/badge/IRC-matterbridgechat-green.svg)](https://webchat.freenode.net/?channels=matterbridgechat) [![Discord](https://img.shields.io/badge/discord-matterbridge-green.svg)](https://discord.gg/AkKPtrQ) [![Matrix](https://img.shields.io/badge/matrix-matterbridge-green.svg)](https://riot.im/app/#/room/#matterbridge:matrix.org)
+Click on one of the badges below to join the chat   
 
-![matterbridge.gif](https://s15.postimg.org/qpjhp6y3f/matterbridge.gif)
+[![Gitter](https://img.shields.io/gitter/room/nwjs/nw.js.svg?colorB=42f4242)](https://gitter.im/42wim/matterbridge) [![Join the IRC chat at https://webchat.freenode.net/?channels=matterbridgechat](https://img.shields.io/badge/IRC-matterbridgechat-green.svg?colorB=42f4242)](https://webchat.freenode.net/?channels=matterbridgechat) [![Discord](https://img.shields.io/badge/discord-matterbridge-green.svg?colorB=42f4242)](https://discord.gg/AkKPtrQ) [![Matrix](https://img.shields.io/badge/matrix-matterbridge-green.svg?colorB=42f4242)](https://riot.im/app/#/room/#matterbridge:matrix.org) [![Slack](https://img.shields.io/badge/slack-matterbridgechat-green.svg?colorB=42f4242)](https://join.slack.com/matterbridgechat/shared_invite/MjEwODMxNjU1NDMwLTE0OTk2MTU3NTMtMzZkZmRiNDZhOA) [![Mattermost](https://img.shields.io/badge/mattermost-matterbridge-green.svg?colorB=42f4242)](https://framateam.org/signup_user_complete/?id=tfqm33ggop8x3qgu4boeieta6e) [![Xmpp](https://img.shields.io/badge/xmpp-matterbridge@conference.jabber.de-green.svg?colorB=42f4242)](https://inverse.chat) [![Twitch](https://img.shields.io/badge/twitch-matterbridge-green.svg?colorB=42f4242)](https://www.twitch.tv/matterbridge) [![Zulip](https://img.shields.io/badge/zulip-matterbridge-green.svg?colorB=42f4242)](https://matterbridge.zulipchat.com/register/)
 
-Simple bridge between Mattermost, IRC, XMPP, Gitter, Slack, Discord, Telegram, Rocket.Chat, Hipchat(via xmpp) and Matrix with REST API.
+[![Download stable](https://img.shields.io/github/release/42wim/matterbridge.svg?label=download%20stable)](https://github.com/42wim/matterbridge/releases/latest) [![Download dev](https://img.shields.io/bintray/v/42wim/nightly/Matterbridge.svg?label=download%20dev&colorB=007ec6)](https://bintray.com/42wim/nightly/Matterbridge/_latestVersion)
+
+![matterbridge.gif](https://github.com/42wim/matterbridge/blob/master/img/matterbridge.gif)
+
+Simple bridge between IRC, XMPP, Gitter, Mattermost, Slack, Discord, Telegram, Rocket.Chat, Hipchat(via xmpp), Matrix, Steam, ssh-chat and Zulip
+Has a REST API.   
+Minecraft server chat support via [MatterLink](https://github.com/elytra/MatterLink)
+
+**Mattermost isn't required to run matterbridge. It bridges between any supported protocol.**   
+(The name matterbridge is a remnant when it was only bridging mattermost)
 
 # Table of Contents
- * [Features](#features)
+ * [Features](https://github.com/42wim/matterbridge/wiki/Features)
  * [Requirements](#requirements)
+ * [Screenshots](https://github.com/42wim/matterbridge/wiki/)
  * [Installing](#installing)
    * [Binaries](#binaries)
    * [Building](#building)
  * [Configuration](#configuration)
+   * [Howto](https://github.com/42wim/matterbridge/wiki/How-to-create-your-config)
    * [Examples](#examples) 
  * [Running](#running)
    * [Docker](#docker)
@@ -20,15 +31,25 @@ Simple bridge between Mattermost, IRC, XMPP, Gitter, Slack, Discord, Telegram, R
  * [Thanks](#thanks)
 
 # Features
-* Relays public channel messages between multiple mattermost, IRC, XMPP, Gitter, Slack, Discord, Telegram, Rocket.Chat, Hipchat (via xmpp) and Matrix. Pick and mix.
-* Matterbridge can also work with private groups on your mattermost/slack.
-* Allow for bridging the same bridges, which means you can eg bridge between multiple mattermosts.
-* The bridge is now a gateway which has support multiple in and out bridges. (and supports multiple gateways).
-* REST API to read/post messages to bridges (WIP).
+* [Support bridging between any protocols](https://github.com/42wim/matterbridge/wiki/Features#support-bridging-between-any-protocols)
+* [Support multiple gateways(bridges) for your protocols](https://github.com/42wim/matterbridge/wiki/Features#support-multiple-gatewaysbridges-for-your-protocols)
+* [Message edits and deletes](https://github.com/42wim/matterbridge/wiki/Features#message-edits-and-deletes)
+* [Attachment / files handling](https://github.com/42wim/matterbridge/wiki/Features#attachment--files-handling)
+* [Username and avatar spoofing](https://github.com/42wim/matterbridge/wiki/Features#username-and-avatar-spoofing)
+* [Private groups](https://github.com/42wim/matterbridge/wiki/Features#private-groups)
+* [API](https://github.com/42wim/matterbridge/wiki/Features#api)
+
+## API
+The API is very basic at the moment and rather undocumented.
+
+Used by at least 2 projects. Feel free to make a PR to add your project to this list.
+
+* [MatterLink](https://github.com/elytra/MatterLink) (Matterbridge link for Minecraft Server chat)
+* [pyCord](https://github.com/NikkyAI/pyCord) (crossplatform chatbot)
 
 # Requirements
 Accounts to one of the supported bridges
-* [Mattermost](https://github.com/mattermost/platform/) 3.5.x - 3.7.x
+* [Mattermost](https://github.com/mattermost/platform/) 3.8.x - 3.10.x, 4.x, 5.x
 * [IRC](http://www.mirc.com/servers.html)
 * [XMPP](https://jabber.org)
 * [Gitter](https://gitter.im)
@@ -38,14 +59,23 @@ Accounts to one of the supported bridges
 * [Hipchat](https://www.hipchat.com)
 * [Rocket.chat](https://rocket.chat)
 * [Matrix](https://matrix.org)
+* [Steam](https://store.steampowered.com/)
+* [Twitch](https://twitch.tv)
+* [Ssh-chat](https://github.com/shazow/ssh-chat)
+* [Zulip](https://zulipchat.com)
+
+# Screenshots
+See https://github.com/42wim/matterbridge/wiki
 
 # Installing
 ## Binaries
-Binaries can be found [here] (https://github.com/42wim/matterbridge/releases/)
-* Latest release [v0.10.1](https://github.com/42wim/matterbridge/releases/latest)
+* Latest stable release [v1.11.0](https://github.com/42wim/matterbridge/releases/latest)
+* Development releases (follows master) can be downloaded [here](https://dl.bintray.com/42wim/nightly/)  
 
 ## Building
-Go 1.6+ is required. Make sure you have [Go](https://golang.org/doc/install) properly installed, including setting up your [GOPATH] (https://golang.org/doc/code.html#GOPATH)
+Go 1.8+ is required. Make sure you have [Go](https://golang.org/doc/install) properly installed, including setting up your [GOPATH](https://golang.org/doc/code.html#GOPATH).
+
+After Go is setup, download matterbridge to your $GOPATH directory. 
 
 ```
 cd $GOPATH
@@ -60,10 +90,13 @@ matterbridge
 ```
 
 # Configuration
-* [matterbridge.toml.sample](https://github.com/42wim/matterbridge/blob/master/matterbridge.toml.sample) for documentation and an example.
-* [matterbridge.toml.simple](https://github.com/42wim/matterbridge/blob/master/matterbridge.toml.simple) for a simple example.
+## Basic configuration
+See [howto](https://github.com/42wim/matterbridge/wiki/How-to-create-your-config) for a step by step walkthrough for creating your configuration.
 
-## Examples
+## Advanced configuration
+* [matterbridge.toml.sample](https://github.com/42wim/matterbridge/blob/master/matterbridge.toml.sample) for documentation and an example.
+
+## Examples 
 ### Bridge mattermost (off-topic) - irc (#testing)
 ```
 [irc]
@@ -73,12 +106,12 @@ matterbridge
 
 [mattermost]
     [mattermost.work]
-    useAPI=true
     Server="yourmattermostserver.tld"
     Team="yourteam"
     Login="yourlogin"
     Password="yourpass"
     PrefixMessagesWithNick=true
+    RemoteNickFormat="[{PROTOCOL}] <{NICK}> "
 
 [[gateway]]
 name="mygateway"
@@ -96,7 +129,6 @@ enable=true
 ```
 [slack]
 [slack.test]
-useAPI=true
 Token="yourslacktoken"
 PrefixMessagesWithNick=true
 
@@ -122,9 +154,8 @@ RemoteNickFormat="[{PROTOCOL}/{BRIDGE}] <{NICK}> "
 ```
 
 # Running
-1) Copy the matterbridge.toml.sample to matterbridge.toml in the same directory as the matterbridge binary.  
-2) Edit matterbridge.toml with the settings for your environment. 
-3) Now you can run matterbridge.  (```./matterbridge```)
+
+See [howto](https://github.com/42wim/matterbridge/wiki/How-to-create-your-config) for a step by step walkthrough for creating your configuration.
 
 ```
 Usage of ./matterbridge:
@@ -132,6 +163,8 @@ Usage of ./matterbridge:
         config file (default "matterbridge.toml")
   -debug
         enable debug
+  -gops
+        enable gops agent
   -version
         show version
 ```
@@ -147,28 +180,26 @@ See [changelog.md](https://github.com/42wim/matterbridge/blob/master/changelog.m
 
 # FAQ
 
-Please look at [matterbridge.toml.sample](https://github.com/42wim/matterbridge/blob/master/matterbridge.toml.sample) for more information first.
-
-## Mattermost doesn't show the IRC nicks
-If you're running the webhooks version, this can be fixed by either:
-* enabling "override usernames". See [mattermost documentation](http://docs.mattermost.com/developer/webhooks-incoming.html#enabling-incoming-webhooks)
-* setting ```PrefixMessagesWithNick``` to ```true``` in ```mattermost``` section of your matterbridge.toml.
-
-If you're running the API version you'll need to:
-* setting ```PrefixMessagesWithNick``` to ```true``` in ```mattermost``` section of your matterbridge.toml.
-
-Also look at the ```RemoteNickFormat``` setting.
+See [FAQ](https://github.com/42wim/matterbridge/wiki/FAQ)
 
+Want to tip ? 
+* eth: 0xb3f9b5387c66ad6be892bcb7bbc67862f3abc16f
+* btc: 1N7cKHj5SfqBHBzDJ6kad4BzeqUBBS2zhs
 
 # Thanks
+[![Digitalocean](https://snag.gy/3LVifX.jpg)](https://www.digitalocean.com/) for sponsoring demo/testing droplets.
+
 Matterbridge wouldn't exist without these libraries:
 * discord - https://github.com/bwmarrin/discordgo
 * echo - https://github.com/labstack/echo
 * gitter - https://github.com/sromku/go-gitter
-* irc - https://github.com/thoj/go-ircevent
+* gops - https://github.com/google/gops
+* gozulipbot - https://github.com/ifo/gozulipbot
+* irc - https://github.com/lrstanley/girc
 * mattermost - https://github.com/mattermost/platform
 * matrix - https://github.com/matrix-org/gomatrix
 * slack - https://github.com/nlopes/slack
+* steam - https://github.com/Philipp15b/go-steam
 * telegram - https://github.com/go-telegram-bot-api/telegram-bot-api
 * xmpp - https://github.com/mattn/go-xmpp
-
+* zulip - https://github.com/ifo/gozulipbot

bridge/api/api.go

@@ -1,47 +1,53 @@
 package api
 
 import (
-	"github.com/42wim/matterbridge/bridge/config"
-	log "github.com/Sirupsen/logrus"
-	"github.com/labstack/echo"
-	"github.com/zfjagann/golang-ring"
+	"encoding/json"
 	"net/http"
 	"sync"
+	"time"
+
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/labstack/echo"
+	"github.com/labstack/echo/middleware"
+	"github.com/zfjagann/golang-ring"
 )
 
 type Api struct {
-	Config   *config.Protocol
-	Remote   chan config.Message
-	Account  string
 	Messages ring.Ring
 	sync.RWMutex
+	*bridge.Config
 }
 
 type ApiMessage struct {
 	Text     string `json:"text"`
 	Username string `json:"username"`
+	UserID   string `json:"userid"`
 	Avatar   string `json:"avatar"`
+	Gateway  string `json:"gateway"`
 }
 
-var flog *log.Entry
-var protocol = "api"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Api {
-	b := &Api{}
+func New(cfg *bridge.Config) bridge.Bridger {
+	b := &Api{Config: cfg}
 	e := echo.New()
+	e.HideBanner = true
+	e.HidePort = true
 	b.Messages = ring.Ring{}
-	b.Messages.SetCapacity(cfg.Buffer)
-	b.Config = &cfg
-	b.Account = account
-	b.Remote = c
+	b.Messages.SetCapacity(b.GetInt("Buffer"))
+	if b.GetString("Token") != "" {
+		e.Use(middleware.KeyAuth(func(key string, c echo.Context) (bool, error) {
+			return key == b.GetString("Token"), nil
+		}))
+	}
 	e.GET("/api/messages", b.handleMessages)
+	e.GET("/api/stream", b.handleStream)
 	e.POST("/api/message", b.handlePostMessage)
 	go func() {
-		flog.Fatal(e.Start(cfg.BindAddress))
+		if b.GetString("BindAddress") == "" {
+			b.Log.Fatalf("No BindAddress configured.")
+		}
+		b.Log.Infof("Listening on %s", b.GetString("BindAddress"))
+		b.Log.Fatal(e.Start(b.GetString("BindAddress")))
 	}()
 	return b
 }
@@ -53,39 +59,63 @@ func (b *Api) Disconnect() error {
 	return nil
 
 }
-func (b *Api) JoinChannel(channel string) error {
+func (b *Api) JoinChannel(channel config.ChannelInfo) error {
 	return nil
 
 }
 
-func (b *Api) Send(msg config.Message) error {
+func (b *Api) Send(msg config.Message) (string, error) {
 	b.Lock()
 	defer b.Unlock()
+	// ignore delete messages
+	if msg.Event == config.EVENT_MSG_DELETE {
+		return "", nil
+	}
 	b.Messages.Enqueue(&msg)
-	return nil
+	return "", nil
 }
 
 func (b *Api) handlePostMessage(c echo.Context) error {
-	message := &ApiMessage{}
-	if err := c.Bind(message); err != nil {
+	message := config.Message{}
+	if err := c.Bind(&message); err != nil {
 		return err
 	}
-	b.Remote <- config.Message{
-		Text:     message.Text,
-		Username: message.Username,
-		Channel:  "api",
-		Avatar:   message.Avatar,
-		Account:  b.Account,
-	}
+	// these values are fixed
+	message.Channel = "api"
+	message.Protocol = "api"
+	message.Account = b.Account
+	message.ID = ""
+	message.Timestamp = time.Now()
+	b.Log.Debugf("Sending message from %s on %s to gateway", message.Username, "api")
+	b.Remote <- message
 	return c.JSON(http.StatusOK, message)
 }
 
 func (b *Api) handleMessages(c echo.Context) error {
 	b.Lock()
 	defer b.Unlock()
-	for _, msg := range b.Messages.Values() {
-		c.JSONPretty(http.StatusOK, msg, " ")
-	}
+	c.JSONPretty(http.StatusOK, b.Messages.Values(), " ")
 	b.Messages = ring.Ring{}
 	return nil
 }
+
+func (b *Api) handleStream(c echo.Context) error {
+	c.Response().Header().Set(echo.HeaderContentType, echo.MIMEApplicationJSON)
+	c.Response().WriteHeader(http.StatusOK)
+	closeNotifier := c.Response().CloseNotify()
+	for {
+		select {
+		case <-closeNotifier:
+			return nil
+		default:
+			msg := b.Messages.Dequeue()
+			if msg != nil {
+				if err := json.NewEncoder(c.Response()).Encode(msg); err != nil {
+					return err
+				}
+				c.Response().Flush()
+			}
+			time.Sleep(200 * time.Millisecond)
+		}
+	}
+}

bridge/bridge.go

@@ -1,116 +1,104 @@
 package bridge
 
 import (
-	"github.com/42wim/matterbridge/bridge/api"
 	"github.com/42wim/matterbridge/bridge/config"
-	"github.com/42wim/matterbridge/bridge/discord"
-	"github.com/42wim/matterbridge/bridge/gitter"
-	"github.com/42wim/matterbridge/bridge/irc"
-	"github.com/42wim/matterbridge/bridge/matrix"
-	"github.com/42wim/matterbridge/bridge/mattermost"
-	"github.com/42wim/matterbridge/bridge/rocketchat"
-	"github.com/42wim/matterbridge/bridge/slack"
-	"github.com/42wim/matterbridge/bridge/telegram"
-	"github.com/42wim/matterbridge/bridge/xmpp"
-	log "github.com/Sirupsen/logrus"
+	log "github.com/sirupsen/logrus"
 
 	"strings"
 )
 
 type Bridger interface {
-	Send(msg config.Message) error
+	Send(msg config.Message) (string, error)
 	Connect() error
-	JoinChannel(channel string) error
+	JoinChannel(channel config.ChannelInfo) error
 	Disconnect() error
 }
 
 type Bridge struct {
-	Config config.Protocol
 	Bridger
-	Name        string
-	Account     string
-	Protocol    string
-	ChannelsIn  map[string]config.ChannelOptions
-	ChannelsOut map[string]config.ChannelOptions
+	Name     string
+	Account  string
+	Protocol string
+	Channels map[string]config.ChannelInfo
+	Joined   map[string]bool
+	Log      *log.Entry
+	Config   *config.Config
+	General  *config.Protocol
 }
 
-func New(cfg *config.Config, bridge *config.Bridge, c chan config.Message) *Bridge {
+type Config struct {
+	//	General *config.Protocol
+	Remote chan config.Message
+	Log    *log.Entry
+	*Bridge
+}
+
+// Factory is the factory function to create a bridge
+type Factory func(*Config) Bridger
+
+func New(bridge *config.Bridge) *Bridge {
 	b := new(Bridge)
-	b.ChannelsIn = make(map[string]config.ChannelOptions)
-	b.ChannelsOut = make(map[string]config.ChannelOptions)
+	b.Channels = make(map[string]config.ChannelInfo)
 	accInfo := strings.Split(bridge.Account, ".")
 	protocol := accInfo[0]
 	name := accInfo[1]
 	b.Name = name
 	b.Protocol = protocol
 	b.Account = bridge.Account
-
-	// override config from environment
-	config.OverrideCfgFromEnv(cfg, protocol, name)
-	switch protocol {
-	case "mattermost":
-		b.Config = cfg.Mattermost[name]
-		b.Bridger = bmattermost.New(cfg.Mattermost[name], bridge.Account, c)
-	case "irc":
-		b.Config = cfg.IRC[name]
-		b.Bridger = birc.New(cfg.IRC[name], bridge.Account, c)
-	case "gitter":
-		b.Config = cfg.Gitter[name]
-		b.Bridger = bgitter.New(cfg.Gitter[name], bridge.Account, c)
-	case "slack":
-		b.Config = cfg.Slack[name]
-		b.Bridger = bslack.New(cfg.Slack[name], bridge.Account, c)
-	case "xmpp":
-		b.Config = cfg.Xmpp[name]
-		b.Bridger = bxmpp.New(cfg.Xmpp[name], bridge.Account, c)
-	case "discord":
-		b.Config = cfg.Discord[name]
-		b.Bridger = bdiscord.New(cfg.Discord[name], bridge.Account, c)
-	case "telegram":
-		b.Config = cfg.Telegram[name]
-		b.Bridger = btelegram.New(cfg.Telegram[name], bridge.Account, c)
-	case "rocketchat":
-		b.Config = cfg.Rocketchat[name]
-		b.Bridger = brocketchat.New(cfg.Rocketchat[name], bridge.Account, c)
-	case "matrix":
-		b.Config = cfg.Matrix[name]
-		b.Bridger = bmatrix.New(cfg.Matrix[name], bridge.Account, c)
-	case "api":
-		b.Config = cfg.Api[name]
-		b.Bridger = api.New(cfg.Api[name], bridge.Account, c)
-	}
+	b.Joined = make(map[string]bool)
 	return b
 }
 
 func (b *Bridge) JoinChannels() error {
-	exists := make(map[string]bool)
-	err := b.joinChannels(b.ChannelsIn, exists)
-	if err != nil {
-		return err
-	}
-	err = b.joinChannels(b.ChannelsOut, exists)
-	if err != nil {
-		return err
-	}
-	return nil
+	err := b.joinChannels(b.Channels, b.Joined)
+	return err
 }
 
-func (b *Bridge) joinChannels(cMap map[string]config.ChannelOptions, exists map[string]bool) error {
-	mychannel := ""
-	for channel, info := range cMap {
-		if !exists[channel] {
-			mychannel = channel
-			log.Infof("%s: joining %s", b.Account, channel)
-			if b.Protocol == "irc" && info.Key != "" {
-				log.Debugf("using key %s for channel %s", info.Key, channel)
-				mychannel = mychannel + " " + info.Key
-			}
-			err := b.JoinChannel(mychannel)
+func (b *Bridge) joinChannels(channels map[string]config.ChannelInfo, exists map[string]bool) error {
+	for ID, channel := range channels {
+		if !exists[ID] {
+			b.Log.Infof("%s: joining %s (ID: %s)", b.Account, channel.Name, ID)
+			err := b.JoinChannel(channel)
 			if err != nil {
 				return err
 			}
-			exists[channel] = true
+			exists[ID] = true
 		}
 	}
 	return nil
 }
+
+func (b *Bridge) GetBool(key string) bool {
+	if b.Config.GetBool(b.Account + "." + key) {
+		return b.Config.GetBool(b.Account + "." + key)
+	}
+	return b.Config.GetBool("general." + key)
+}
+
+func (b *Bridge) GetInt(key string) int {
+	if b.Config.GetInt(b.Account+"."+key) != 0 {
+		return b.Config.GetInt(b.Account + "." + key)
+	}
+	return b.Config.GetInt("general." + key)
+}
+
+func (b *Bridge) GetString(key string) string {
+	if b.Config.GetString(b.Account+"."+key) != "" {
+		return b.Config.GetString(b.Account + "." + key)
+	}
+	return b.Config.GetString("general." + key)
+}
+
+func (b *Bridge) GetStringSlice(key string) []string {
+	if len(b.Config.GetStringSlice(b.Account+"."+key)) != 0 {
+		return b.Config.GetStringSlice(b.Account + "." + key)
+	}
+	return b.Config.GetStringSlice("general." + key)
+}
+
+func (b *Bridge) GetStringSlice2D(key string) [][]string {
+	if len(b.Config.GetStringSlice2D(b.Account+"."+key)) != 0 {
+		return b.Config.GetStringSlice2D(b.Account + "." + key)
+	}
+	return b.Config.GetStringSlice2D("general." + key)
+}

bridge/config/config.go

@@ -1,71 +1,140 @@
 package config
 
 import (
-	"github.com/BurntSushi/toml"
-	"log"
+	"bytes"
 	"os"
-	"reflect"
 	"strings"
+	"sync"
 	"time"
+
+	"github.com/fsnotify/fsnotify"
+	log "github.com/sirupsen/logrus"
+	"github.com/spf13/viper"
+	prefixed "github.com/x-cray/logrus-prefixed-formatter"
 )
 
 const (
-	EVENT_JOIN_LEAVE = "join_leave"
-	EVENT_FAILURE    = "failure"
+	EVENT_JOIN_LEAVE        = "join_leave"
+	EVENT_TOPIC_CHANGE      = "topic_change"
+	EVENT_FAILURE           = "failure"
+	EVENT_FILE_FAILURE_SIZE = "file_failure_size"
+	EVENT_AVATAR_DOWNLOAD   = "avatar_download"
+	EVENT_REJOIN_CHANNELS   = "rejoin_channels"
+	EVENT_USER_ACTION       = "user_action"
+	EVENT_MSG_DELETE        = "msg_delete"
 )
 
 type Message struct {
-	Text      string
-	Channel   string
-	Username  string
-	Avatar    string
-	Account   string
-	Event     string
-	Protocol  string
-	Timestamp time.Time
+	Text      string    `json:"text"`
+	Channel   string    `json:"channel"`
+	Username  string    `json:"username"`
+	UserID    string    `json:"userid"` // userid on the bridge
+	Avatar    string    `json:"avatar"`
+	Account   string    `json:"account"`
+	Event     string    `json:"event"`
+	Protocol  string    `json:"protocol"`
+	Gateway   string    `json:"gateway"`
+	Timestamp time.Time `json:"timestamp"`
+	ID        string    `json:"id"`
+	Extra     map[string][]interface{}
+}
+
+type FileInfo struct {
+	Name    string
+	Data    *[]byte
+	Comment string
+	URL     string
+	Size    int64
+	Avatar  bool
+	SHA     string
+}
+
+type ChannelInfo struct {
+	Name        string
+	Account     string
+	Direction   string
+	ID          string
+	SameChannel map[string]bool
+	Options     ChannelOptions
 }
 
 type Protocol struct {
-	BindAddress            string // mattermost, slack
+	AuthCode               string // steam
+	BindAddress            string // mattermost, slack // DEPRECATED
 	Buffer                 int    // api
+	Charset                string // irc
+	ColorNicks             bool   // only irc for now
+	Debug                  bool   // general
+	DebugLevel             int    // only for irc now
+	EditSuffix             string // mattermost, slack, discord, telegram, gitter
+	EditDisable            bool   // mattermost, slack, discord, telegram, gitter
 	IconURL                string // mattermost, slack
 	IgnoreNicks            string // all protocols
+	IgnoreMessages         string // all protocols
 	Jid                    string // xmpp
+	Label                  string // all protocols
 	Login                  string // mattermost, matrix
-	Muc                    string // xmpp
-	Name                   string // all protocols
-	Nick                   string // all protocols
-	NickFormatter          string // mattermost, slack
-	NickServNick           string // IRC
-	NickServPassword       string // IRC
-	NicksPerRow            int    // mattermost, slack
-	NoTLS                  bool   // mattermost
-	Password               string // IRC,mattermost,XMPP,matrix
-	PrefixMessagesWithNick bool   // mattemost, slack
-	Protocol               string //all protocols
-	MessageQueue           int    // IRC, size of message queue for flood control
-	MessageDelay           int    // IRC, time in millisecond to wait between messages
-	MessageFormat          string // telegram
-	RemoteNickFormat       string // all protocols
-	Server                 string // IRC,mattermost,XMPP,discord
-	ShowJoinPart           bool   // all protocols
-	SkipTLSVerify          bool   // IRC, mattermost
-	Team                   string // mattermost
-	Token                  string // gitter, slack, discord
-	URL                    string // mattermost, slack, matrix
-	UseAPI                 bool   // mattermost, slack
-	UseSASL                bool   // IRC
-	UseTLS                 bool   // IRC
+	MediaDownloadBlackList []string
+	MediaDownloadPath      string // Basically MediaServerUpload, but instead of uploading it, just write it to a file on the same server.
+	MediaDownloadSize      int    // all protocols
+	MediaServerDownload    string
+	MediaServerUpload      string
+	MessageDelay           int        // IRC, time in millisecond to wait between messages
+	MessageFormat          string     // telegram
+	MessageLength          int        // IRC, max length of a message allowed
+	MessageQueue           int        // IRC, size of message queue for flood control
+	MessageSplit           bool       // IRC, split long messages with newlines on MessageLength instead of clipping
+	Muc                    string     // xmpp
+	Name                   string     // all protocols
+	Nick                   string     // all protocols
+	NickFormatter          string     // mattermost, slack
+	NickServNick           string     // IRC
+	NickServUsername       string     // IRC
+	NickServPassword       string     // IRC
+	NicksPerRow            int        // mattermost, slack
+	NoHomeServerSuffix     bool       // matrix
+	NoSendJoinPart         bool       // all protocols
+	NoTLS                  bool       // mattermost
+	Password               string     // IRC,mattermost,XMPP,matrix
+	PrefixMessagesWithNick bool       // mattemost, slack
+	Protocol               string     // all protocols
+	QuoteDisable           bool       // telegram
+	QuoteFormat            string     // telegram
+	RejoinDelay            int        // IRC
+	ReplaceMessages        [][]string // all protocols
+	ReplaceNicks           [][]string // all protocols
+	RemoteNickFormat       string     // all protocols
+	Server                 string     // IRC,mattermost,XMPP,discord
+	ShowJoinPart           bool       // all protocols
+	ShowTopicChange        bool       // slack
+	ShowEmbeds             bool       // discord
+	SkipTLSVerify          bool       // IRC, mattermost
+	StripNick              bool       // all protocols
+	Team                   string     // mattermost
+	Token                  string     // gitter, slack, discord, api
+	Topic                  string     // zulip
+	URL                    string     // mattermost, slack // DEPRECATED
+	UseAPI                 bool       // mattermost, slack
+	UseSASL                bool       // IRC
+	UseTLS                 bool       // IRC
+	UseFirstName           bool       // telegram
+	UseUserName            bool       // discord
+	UseInsecureURL         bool       // telegram
+	WebhookBindAddress     string     // mattermost, slack
+	WebhookURL             string     // mattermost, slack
+	WebhookUse             string     // mattermost, slack, discord
 }
 
 type ChannelOptions struct {
-	Key string // irc
+	Key        string // irc, xmpp
+	WebhookURL string // discord
 }
 
 type Bridge struct {
-	Account string
-	Channel string
-	Options ChannelOptions
+	Account     string
+	Channel     string
+	Options     ChannelOptions
+	SameChannel bool
 }
 
 type Gateway struct {
@@ -83,69 +152,130 @@ type SameChannelGateway struct {
 	Accounts []string
 }
 
-type Config struct {
+type ConfigValues struct {
 	Api                map[string]Protocol
-	IRC                map[string]Protocol
+	Irc                map[string]Protocol
 	Mattermost         map[string]Protocol
 	Matrix             map[string]Protocol
 	Slack              map[string]Protocol
+	Steam              map[string]Protocol
 	Gitter             map[string]Protocol
 	Xmpp               map[string]Protocol
 	Discord            map[string]Protocol
 	Telegram           map[string]Protocol
 	Rocketchat         map[string]Protocol
+	Sshchat            map[string]Protocol
+	Zulip              map[string]Protocol
 	General            Protocol
 	Gateway            []Gateway
 	SameChannelGateway []SameChannelGateway
 }
 
+type Config struct {
+	v *viper.Viper
+	*ConfigValues
+	sync.RWMutex
+}
+
 func NewConfig(cfgfile string) *Config {
-	var cfg Config
-	if _, err := toml.DecodeFile(cfgfile, &cfg); err != nil {
+	log.SetFormatter(&prefixed.TextFormatter{PrefixPadding: 13, DisableColors: true, FullTimestamp: false})
+	flog := log.WithFields(log.Fields{"prefix": "config"})
+	var cfg ConfigValues
+	viper.SetConfigType("toml")
+	viper.SetConfigFile(cfgfile)
+	viper.SetEnvPrefix("matterbridge")
+	viper.AddConfigPath(".")
+	viper.SetEnvKeyReplacer(strings.NewReplacer(".", "_"))
+	viper.AutomaticEnv()
+	f, err := os.Open(cfgfile)
+	if err != nil {
 		log.Fatal(err)
 	}
-	return &cfg
-}
-
-func OverrideCfgFromEnv(cfg *Config, protocol string, account string) {
-	var protoCfg Protocol
-	val := reflect.ValueOf(cfg).Elem()
-	// loop over the Config struct
-	for i := 0; i < val.NumField(); i++ {
-		typeField := val.Type().Field(i)
-		// look for the protocol map (both lowercase)
-		if strings.ToLower(typeField.Name) == protocol {
-			// get the Protocol struct from the map
-			data := val.Field(i).MapIndex(reflect.ValueOf(account))
-			protoCfg = data.Interface().(Protocol)
-			protoStruct := reflect.ValueOf(&protoCfg).Elem()
-			// loop over the found protocol struct
-			for i := 0; i < protoStruct.NumField(); i++ {
-				typeField := protoStruct.Type().Field(i)
-				// build our environment key (eg MATTERBRIDGE_MATTERMOST_WORK_LOGIN)
-				key := "matterbridge_" + protocol + "_" + account + "_" + typeField.Name
-				key = strings.ToUpper(key)
-				// search the environment
-				res := os.Getenv(key)
-				// if it exists and the current field is a string
-				// then update the current field
-				if res != "" {
-					fieldVal := protoStruct.Field(i)
-					if fieldVal.Kind() == reflect.String {
-						log.Printf("config: overriding %s from env with %s\n", key, res)
-						fieldVal.Set(reflect.ValueOf(res))
-					}
-				}
-			}
-			// update the map with the modified Protocol (cfg.Protocol[account] = Protocol)
-			val.Field(i).SetMapIndex(reflect.ValueOf(account), reflect.ValueOf(protoCfg))
-			break
-		}
+	err = viper.ReadConfig(f)
+	if err != nil {
+		log.Fatal(err)
 	}
+	err = viper.Unmarshal(&cfg)
+	if err != nil {
+		log.Fatal("blah", err)
+	}
+	mycfg := new(Config)
+	mycfg.v = viper.GetViper()
+	if cfg.General.MediaDownloadSize == 0 {
+		cfg.General.MediaDownloadSize = 1000000
+	}
+	viper.WatchConfig()
+	viper.OnConfigChange(func(e fsnotify.Event) {
+		flog.Println("Config file changed:", e.Name)
+	})
+
+	mycfg.ConfigValues = &cfg
+	return mycfg
 }
 
-func GetIconURL(msg *Message, cfg *Protocol) string {
-	iconURL := cfg.IconURL
+func NewConfigFromString(input []byte) *Config {
+	var cfg ConfigValues
+	viper.SetConfigType("toml")
+	err := viper.ReadConfig(bytes.NewBuffer(input))
+	if err != nil {
+		log.Fatal(err)
+	}
+	err = viper.Unmarshal(&cfg)
+	if err != nil {
+		log.Fatal(err)
+	}
+	mycfg := new(Config)
+	mycfg.v = viper.GetViper()
+	mycfg.ConfigValues = &cfg
+	return mycfg
+}
+
+func (c *Config) GetBool(key string) bool {
+	c.RLock()
+	defer c.RUnlock()
+	//	log.Debugf("getting bool %s = %#v", key, c.v.GetBool(key))
+	return c.v.GetBool(key)
+}
+
+func (c *Config) GetInt(key string) int {
+	c.RLock()
+	defer c.RUnlock()
+	//	log.Debugf("getting int %s = %d", key, c.v.GetInt(key))
+	return c.v.GetInt(key)
+}
+
+func (c *Config) GetString(key string) string {
+	c.RLock()
+	defer c.RUnlock()
+	//	log.Debugf("getting String %s = %s", key, c.v.GetString(key))
+	return c.v.GetString(key)
+}
+
+func (c *Config) GetStringSlice(key string) []string {
+	c.RLock()
+	defer c.RUnlock()
+	// log.Debugf("getting StringSlice %s = %#v", key, c.v.GetStringSlice(key))
+	return c.v.GetStringSlice(key)
+}
+
+func (c *Config) GetStringSlice2D(key string) [][]string {
+	c.RLock()
+	defer c.RUnlock()
+	result := [][]string{}
+	if res, ok := c.v.Get(key).([]interface{}); ok {
+		for _, entry := range res {
+			result2 := []string{}
+			for _, entry2 := range entry.([]interface{}) {
+				result2 = append(result2, entry2.(string))
+			}
+			result = append(result, result2)
+		}
+		return result
+	}
+	return result
+}
+
+func GetIconURL(msg *Message, iconURL string) string {
 	info := strings.Split(msg.Account, ".")
 	protocol := info[0]
 	name := info[1]

bridge/discord/discord.go

@@ -1,160 +1,311 @@
 package bdiscord
 
 import (
-	"github.com/42wim/matterbridge/bridge/config"
-	log "github.com/Sirupsen/logrus"
-	"github.com/bwmarrin/discordgo"
+	"bytes"
+	"fmt"
+	"regexp"
 	"strings"
 	"sync"
+
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
+	"github.com/bwmarrin/discordgo"
 )
 
-type bdiscord struct {
-	c             *discordgo.Session
-	Config        *config.Protocol
-	Remote        chan config.Message
-	Account       string
-	Channels      []*discordgo.Channel
-	Nick          string
-	UseChannelID  bool
-	userMemberMap map[string]*discordgo.Member
-	guildID       string
+type Bdiscord struct {
+	c              *discordgo.Session
+	Channels       []*discordgo.Channel
+	Nick           string
+	UseChannelID   bool
+	userMemberMap  map[string]*discordgo.Member
+	guildID        string
+	webhookID      string
+	webhookToken   string
+	channelInfoMap map[string]*config.ChannelInfo
 	sync.RWMutex
+	*bridge.Config
 }
 
-var flog *log.Entry
-var protocol = "discord"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *bdiscord {
-	b := &bdiscord{}
-	b.Config = &cfg
-	b.Remote = c
-	b.Account = account
+func New(cfg *bridge.Config) bridge.Bridger {
+	b := &Bdiscord{Config: cfg}
 	b.userMemberMap = make(map[string]*discordgo.Member)
+	b.channelInfoMap = make(map[string]*config.ChannelInfo)
+	if b.GetString("WebhookURL") != "" {
+		b.Log.Debug("Configuring Discord Incoming Webhook")
+		b.webhookID, b.webhookToken = b.splitURL(b.GetString("WebhookURL"))
+	}
 	return b
 }
 
-func (b *bdiscord) Connect() error {
+func (b *Bdiscord) Connect() error {
 	var err error
-	flog.Info("Connecting")
-	if !strings.HasPrefix(b.Config.Token, "Bot ") {
-		b.Config.Token = "Bot " + b.Config.Token
+	var token string
+	b.Log.Info("Connecting")
+	if b.GetString("WebhookURL") == "" {
+		b.Log.Info("Connecting using token")
+	} else {
+		b.Log.Info("Connecting using webhookurl (for posting) and token")
 	}
-	b.c, err = discordgo.New(b.Config.Token)
+	if !strings.HasPrefix(b.GetString("Token"), "Bot ") {
+		token = "Bot " + b.GetString("Token")
+	}
+	b.c, err = discordgo.New(token)
 	if err != nil {
-		flog.Debugf("%#v", err)
 		return err
 	}
-	flog.Info("Connection succeeded")
+	b.Log.Info("Connection succeeded")
 	b.c.AddHandler(b.messageCreate)
 	b.c.AddHandler(b.memberUpdate)
+	b.c.AddHandler(b.messageUpdate)
+	b.c.AddHandler(b.messageDelete)
 	err = b.c.Open()
 	if err != nil {
-		flog.Debugf("%#v", err)
 		return err
 	}
-	guilds, err := b.c.UserGuilds()
+	guilds, err := b.c.UserGuilds(100, "", "")
 	if err != nil {
-		flog.Debugf("%#v", err)
 		return err
 	}
 	userinfo, err := b.c.User("@me")
 	if err != nil {
-		flog.Debugf("%#v", err)
 		return err
 	}
 	b.Nick = userinfo.Username
 	for _, guild := range guilds {
-		if guild.Name == b.Config.Server {
+		if guild.Name == b.GetString("Server") {
 			b.Channels, err = b.c.GuildChannels(guild.ID)
 			b.guildID = guild.ID
 			if err != nil {
-				flog.Debugf("%#v", err)
 				return err
 			}
 		}
 	}
+	for _, channel := range b.Channels {
+		b.Log.Debugf("found channel %#v", channel)
+	}
 	return nil
 }
 
-func (b *bdiscord) Disconnect() error {
-	return nil
+func (b *Bdiscord) Disconnect() error {
+	return b.c.Close()
 }
 
-func (b *bdiscord) JoinChannel(channel string) error {
-	idcheck := strings.Split(channel, "ID:")
+func (b *Bdiscord) JoinChannel(channel config.ChannelInfo) error {
+	b.channelInfoMap[channel.ID] = &channel
+	idcheck := strings.Split(channel.Name, "ID:")
 	if len(idcheck) > 1 {
 		b.UseChannelID = true
 	}
 	return nil
 }
 
-func (b *bdiscord) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
+func (b *Bdiscord) Send(msg config.Message) (string, error) {
+	b.Log.Debugf("=> Receiving %#v", msg)
+
 	channelID := b.getChannelID(msg.Channel)
 	if channelID == "" {
-		flog.Errorf("Could not find channelID for %v", msg.Channel)
-		return nil
+		return "", fmt.Errorf("Could not find channelID for %v", msg.Channel)
 	}
-	b.c.ChannelMessageSend(channelID, msg.Username+msg.Text)
-	return nil
+
+	// Make a action /me of the message
+	if msg.Event == config.EVENT_USER_ACTION {
+		msg.Text = "_" + msg.Text + "_"
+	}
+
+	// use initial webhook
+	wID := b.webhookID
+	wToken := b.webhookToken
+
+	// check if have a channel specific webhook
+	if ci, ok := b.channelInfoMap[msg.Channel+b.Account]; ok {
+		if ci.Options.WebhookURL != "" {
+			wID, wToken = b.splitURL(ci.Options.WebhookURL)
+		}
+	}
+
+	// Use webhook to send the message
+	if wID != "" {
+		// skip events
+		if msg.Event != "" {
+			return "", nil
+		}
+		b.Log.Debugf("Broadcasting using Webhook")
+		for _, f := range msg.Extra["file"] {
+			fi := f.(config.FileInfo)
+			if fi.URL != "" {
+				msg.Text += fi.URL + " "
+			}
+		}
+		err := b.c.WebhookExecute(
+			wID,
+			wToken,
+			true,
+			&discordgo.WebhookParams{
+				Content:   msg.Text,
+				Username:  msg.Username,
+				AvatarURL: msg.Avatar,
+			})
+		return "", err
+	}
+
+	b.Log.Debugf("Broadcasting using token (API)")
+
+	// Delete message
+	if msg.Event == config.EVENT_MSG_DELETE {
+		if msg.ID == "" {
+			return "", nil
+		}
+		err := b.c.ChannelMessageDelete(channelID, msg.ID)
+		return "", err
+	}
+
+	// Upload a file if it exists
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.c.ChannelMessageSend(channelID, rmsg.Username+rmsg.Text)
+		}
+		// check if we have files to upload (from slack, telegram or mattermost)
+		if len(msg.Extra["file"]) > 0 {
+			return b.handleUploadFile(&msg, channelID)
+		}
+	}
+
+	// Edit message
+	if msg.ID != "" {
+		_, err := b.c.ChannelMessageEdit(channelID, msg.ID, msg.Username+msg.Text)
+		return msg.ID, err
+	}
+
+	// Post normal message
+	res, err := b.c.ChannelMessageSend(channelID, msg.Username+msg.Text)
+	if err != nil {
+		return "", err
+	}
+	return res.ID, err
 }
 
-func (b *bdiscord) messageCreate(s *discordgo.Session, m *discordgo.MessageCreate) {
+func (b *Bdiscord) messageDelete(s *discordgo.Session, m *discordgo.MessageDelete) {
+	rmsg := config.Message{Account: b.Account, ID: m.ID, Event: config.EVENT_MSG_DELETE, Text: config.EVENT_MSG_DELETE}
+	rmsg.Channel = b.getChannelName(m.ChannelID)
+	if b.UseChannelID {
+		rmsg.Channel = "ID:" + m.ChannelID
+	}
+	b.Log.Debugf("<= Sending message from %s to gateway", b.Account)
+	b.Log.Debugf("<= Message is %#v", rmsg)
+	b.Remote <- rmsg
+}
+
+func (b *Bdiscord) messageUpdate(s *discordgo.Session, m *discordgo.MessageUpdate) {
+	if b.GetBool("EditDisable") {
+		return
+	}
+	// only when message is actually edited
+	if m.Message.EditedTimestamp != "" {
+		b.Log.Debugf("Sending edit message")
+		m.Content = m.Content + b.GetString("EditSuffix")
+		b.messageCreate(s, (*discordgo.MessageCreate)(m))
+	}
+}
+
+func (b *Bdiscord) messageCreate(s *discordgo.Session, m *discordgo.MessageCreate) {
+	var err error
+
 	// not relay our own messages
 	if m.Author.Username == b.Nick {
 		return
 	}
+	// if using webhooks, do not relay if it's ours
+	if b.useWebhook() && m.Author.Bot && b.isWebhookID(m.Author.ID) {
+		return
+	}
+
+	// add the url of the attachments to content
 	if len(m.Attachments) > 0 {
 		for _, attach := range m.Attachments {
 			m.Content = m.Content + "\n" + attach.URL
 		}
 	}
-	if m.Content == "" {
+
+	rmsg := config.Message{Account: b.Account, Avatar: "https://cdn.discordapp.com/avatars/" + m.Author.ID + "/" + m.Author.Avatar + ".jpg", UserID: m.Author.ID, ID: m.ID}
+
+	if m.Content != "" {
+		b.Log.Debugf("== Receiving event %#v", m.Message)
+		m.Message.Content = b.stripCustomoji(m.Message.Content)
+		m.Message.Content = b.replaceChannelMentions(m.Message.Content)
+		rmsg.Text, err = m.ContentWithMoreMentionsReplaced(b.c)
+		if err != nil {
+			b.Log.Errorf("ContentWithMoreMentionsReplaced failed: %s", err)
+			rmsg.Text = m.ContentWithMentionsReplaced()
+		}
+	}
+
+	// set channel name
+	rmsg.Channel = b.getChannelName(m.ChannelID)
+	if b.UseChannelID {
+		rmsg.Channel = "ID:" + m.ChannelID
+	}
+
+	// set username
+	if !b.GetBool("UseUserName") {
+		rmsg.Username = b.getNick(m.Author)
+	} else {
+		rmsg.Username = m.Author.Username
+	}
+
+	// if we have embedded content add it to text
+	if b.GetBool("ShowEmbeds") && m.Message.Embeds != nil {
+		for _, embed := range m.Message.Embeds {
+			rmsg.Text = rmsg.Text + "embed: " + embed.Title + " - " + embed.Description + " - " + embed.URL + "\n"
+		}
+	}
+
+	// no empty messages
+	if rmsg.Text == "" {
 		return
 	}
-	flog.Debugf("Sending message from %s on %s to gateway", m.Author.Username, b.Account)
-	channelName := b.getChannelName(m.ChannelID)
-	if b.UseChannelID {
-		channelName = "ID:" + m.ChannelID
+
+	// do we have a /me action
+	var ok bool
+	rmsg.Text, ok = b.replaceAction(rmsg.Text)
+	if ok {
+		rmsg.Event = config.EVENT_USER_ACTION
 	}
-	username := b.getNick(m.Author)
-	if len(m.MentionRoles) > 0 {
-		m.Message.Content = b.replaceRoleMentions(m.Message.Content)
-	}
-	b.Remote <- config.Message{Username: username, Text: m.ContentWithMentionsReplaced(), Channel: channelName,
-		Account: b.Account, Avatar: "https://cdn.discordapp.com/avatars/" + m.Author.ID + "/" + m.Author.Avatar + ".jpg"}
+
+	b.Log.Debugf("<= Sending message from %s on %s to gateway", m.Author.Username, b.Account)
+	b.Log.Debugf("<= Message is %#v", rmsg)
+	b.Remote <- rmsg
 }
 
-func (b *bdiscord) memberUpdate(s *discordgo.Session, m *discordgo.GuildMemberUpdate) {
+func (b *Bdiscord) memberUpdate(s *discordgo.Session, m *discordgo.GuildMemberUpdate) {
 	b.Lock()
 	if _, ok := b.userMemberMap[m.Member.User.ID]; ok {
-		flog.Debugf("%s: memberupdate: user %s (nick %s) changes nick to %s", b.Account, m.Member.User.Username, b.userMemberMap[m.Member.User.ID].Nick, m.Member.Nick)
+		b.Log.Debugf("%s: memberupdate: user %s (nick %s) changes nick to %s", b.Account, m.Member.User.Username, b.userMemberMap[m.Member.User.ID].Nick, m.Member.Nick)
 	}
 	b.userMemberMap[m.Member.User.ID] = m.Member
 	b.Unlock()
 }
 
-func (b *bdiscord) getNick(user *discordgo.User) string {
+func (b *Bdiscord) getNick(user *discordgo.User) string {
 	var err error
 	b.Lock()
 	defer b.Unlock()
 	if _, ok := b.userMemberMap[user.ID]; ok {
-		if b.userMemberMap[user.ID].Nick != "" {
-			// only return if nick is set
-			return b.userMemberMap[user.ID].Nick
+		if b.userMemberMap[user.ID] != nil {
+			if b.userMemberMap[user.ID].Nick != "" {
+				// only return if nick is set
+				return b.userMemberMap[user.ID].Nick
+			}
+			// otherwise return username
+			return user.Username
 		}
-		// otherwise return username
-		return user.Username
 	}
 	// if we didn't find nick, search for it
-	b.userMemberMap[user.ID], err = b.c.GuildMember(b.guildID, user.ID)
+	member, err := b.c.GuildMember(b.guildID, user.ID)
 	if err != nil {
 		return user.Username
 	}
+	b.userMemberMap[user.ID] = member
 	// only return if nick is set
 	if b.userMemberMap[user.ID].Nick != "" {
 		return b.userMemberMap[user.ID].Nick
@@ -162,7 +313,7 @@ func (b *bdiscord) getNick(user *discordgo.User) string {
 	return user.Username
 }
 
-func (b *bdiscord) getChannelID(name string) string {
+func (b *Bdiscord) getChannelID(name string) string {
 	idcheck := strings.Split(name, "ID:")
 	if len(idcheck) > 1 {
 		return idcheck[1]
@@ -175,7 +326,7 @@ func (b *bdiscord) getChannelID(name string) string {
 	return ""
 }
 
-func (b *bdiscord) getChannelName(id string) string {
+func (b *Bdiscord) getChannelName(id string) string {
 	for _, channel := range b.Channels {
 		if channel.ID == id {
 			return channel.Name
@@ -184,14 +335,90 @@ func (b *bdiscord) getChannelName(id string) string {
 	return ""
 }
 
-func (b *bdiscord) replaceRoleMentions(text string) string {
-	roles, err := b.c.GuildRoles(b.guildID)
-	if err != nil {
-		flog.Debugf("%#v", string(err.(*discordgo.RESTError).ResponseBody))
-		return text
-	}
-	for _, role := range roles {
-		text = strings.Replace(text, "<@&"+role.ID+">", "@"+role.Name, -1)
-	}
+func (b *Bdiscord) replaceChannelMentions(text string) string {
+	var err error
+	re := regexp.MustCompile("<#[0-9]+>")
+	text = re.ReplaceAllStringFunc(text, func(m string) string {
+		channel := b.getChannelName(m[2 : len(m)-1])
+		// if at first don't succeed, try again
+		if channel == "" {
+			b.Channels, err = b.c.GuildChannels(b.guildID)
+			if err != nil {
+				return "#unknownchannel"
+			}
+			channel = b.getChannelName(m[2 : len(m)-1])
+			return "#" + channel
+		}
+		return "#" + channel
+	})
 	return text
 }
+
+func (b *Bdiscord) replaceAction(text string) (string, bool) {
+	if strings.HasPrefix(text, "_") && strings.HasSuffix(text, "_") {
+		return strings.Replace(text, "_", "", -1), true
+	}
+	return text, false
+}
+
+func (b *Bdiscord) stripCustomoji(text string) string {
+	// <:doge:302803592035958784>
+	re := regexp.MustCompile("<(:.*?:)[0-9]+>")
+	return re.ReplaceAllString(text, `$1`)
+}
+
+// splitURL splits a webhookURL and returns the id and token
+func (b *Bdiscord) splitURL(url string) (string, string) {
+	webhookURLSplit := strings.Split(url, "/")
+	if len(webhookURLSplit) != 7 {
+		b.Log.Fatalf("%s is no correct discord WebhookURL", url)
+	}
+	return webhookURLSplit[len(webhookURLSplit)-2], webhookURLSplit[len(webhookURLSplit)-1]
+}
+
+// useWebhook returns true if we have a webhook defined somewhere
+func (b *Bdiscord) useWebhook() bool {
+	if b.GetString("WebhookURL") != "" {
+		return true
+	}
+	for _, channel := range b.channelInfoMap {
+		if channel.Options.WebhookURL != "" {
+			return true
+		}
+	}
+	return false
+}
+
+// isWebhookID returns true if the specified id is used in a defined webhook
+func (b *Bdiscord) isWebhookID(id string) bool {
+	if b.GetString("WebhookURL") != "" {
+		wID, _ := b.splitURL(b.GetString("WebhookURL"))
+		if wID == id {
+			return true
+		}
+	}
+	for _, channel := range b.channelInfoMap {
+		if channel.Options.WebhookURL != "" {
+			wID, _ := b.splitURL(channel.Options.WebhookURL)
+			if wID == id {
+				return true
+			}
+		}
+	}
+	return false
+}
+
+// handleUploadFile handles native upload of files
+func (b *Bdiscord) handleUploadFile(msg *config.Message, channelID string) (string, error) {
+	var err error
+	for _, f := range msg.Extra["file"] {
+		fi := f.(config.FileInfo)
+		files := []*discordgo.File{}
+		files = append(files, &discordgo.File{fi.Name, "", bytes.NewReader(*fi.Data)})
+		_, err = b.c.ChannelMessageSendComplex(channelID, &discordgo.MessageSend{Content: msg.Username + fi.Comment, Files: files})
+		if err != nil {
+			return "", fmt.Errorf("file upload failed: %#v", err)
+		}
+	}
+	return "", nil
+}

bridge/gitter/gitter.go

@@ -2,47 +2,39 @@ package bgitter
 
 import (
 	"fmt"
-	"github.com/42wim/matterbridge/bridge/config"
-	log "github.com/Sirupsen/logrus"
-	"github.com/sromku/go-gitter"
 	"strings"
+
+	"github.com/42wim/go-gitter"
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
 )
 
 type Bgitter struct {
-	c       *gitter.Gitter
-	Config  *config.Protocol
-	Remote  chan config.Message
-	Account string
-	Users   []gitter.User
-	Rooms   []gitter.Room
+	c     *gitter.Gitter
+	User  *gitter.User
+	Users []gitter.User
+	Rooms []gitter.Room
+	*bridge.Config
 }
 
-var flog *log.Entry
-var protocol = "gitter"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Bgitter {
-	b := &Bgitter{}
-	b.Config = &cfg
-	b.Remote = c
-	b.Account = account
-	return b
+func New(cfg *bridge.Config) bridge.Bridger {
+	return &Bgitter{Config: cfg}
 }
 
 func (b *Bgitter) Connect() error {
 	var err error
-	flog.Info("Connecting")
-	b.c = gitter.New(b.Config.Token)
-	_, err = b.c.GetUser()
+	b.Log.Info("Connecting")
+	b.c = gitter.New(b.GetString("Token"))
+	b.User, err = b.c.GetUser()
 	if err != nil {
-		flog.Debugf("%#v", err)
 		return err
 	}
-	flog.Info("Connection succeeded")
-	b.Rooms, _ = b.c.GetRooms()
+	b.Rooms, err = b.c.GetRooms()
+	if err != nil {
+		return err
+	}
+	b.Log.Info("Connection succeeded")
 	return nil
 }
 
@@ -51,10 +43,10 @@ func (b *Bgitter) Disconnect() error {
 
 }
 
-func (b *Bgitter) JoinChannel(channel string) error {
-	roomID, err := b.c.GetRoomId(channel)
+func (b *Bgitter) JoinChannel(channel config.ChannelInfo) error {
+	roomID, err := b.c.GetRoomId(channel.Name)
 	if err != nil {
-		return fmt.Errorf("Could not find roomID for %v. Please create the room on gitter.im", channel)
+		return fmt.Errorf("Could not find roomID for %v. Please create the room on gitter.im", channel.Name)
 	}
 	room, err := b.c.GetRoom(roomID)
 	if err != nil {
@@ -78,29 +70,74 @@ func (b *Bgitter) JoinChannel(channel string) error {
 		for event := range stream.Event {
 			switch ev := event.Data.(type) {
 			case *gitter.MessageReceived:
-				// check for ZWSP to see if it's not an echo
-				if !strings.HasSuffix(ev.Message.Text, "​") {
-					flog.Debugf("Sending message from %s on %s to gateway", ev.Message.From.Username, b.Account)
-					b.Remote <- config.Message{Username: ev.Message.From.Username, Text: ev.Message.Text, Channel: room,
-						Account: b.Account, Avatar: b.getAvatar(ev.Message.From.Username)}
+				// ignore message sent from ourselves
+				if ev.Message.From.ID != b.User.ID {
+					b.Log.Debugf("<= Sending message from %s on %s to gateway", ev.Message.From.Username, b.Account)
+					rmsg := config.Message{Username: ev.Message.From.Username, Text: ev.Message.Text, Channel: room,
+						Account: b.Account, Avatar: b.getAvatar(ev.Message.From.Username), UserID: ev.Message.From.ID,
+						ID: ev.Message.ID}
+					if strings.HasPrefix(ev.Message.Text, "@"+ev.Message.From.Username) {
+						rmsg.Event = config.EVENT_USER_ACTION
+						rmsg.Text = strings.Replace(rmsg.Text, "@"+ev.Message.From.Username+" ", "", -1)
+					}
+					b.Log.Debugf("<= Message is %#v", rmsg)
+					b.Remote <- rmsg
 				}
 			case *gitter.GitterConnectionClosed:
-				flog.Errorf("connection with gitter closed for room %s", room)
+				b.Log.Errorf("connection with gitter closed for room %s", room)
 			}
 		}
-	}(stream, room.Name)
+	}(stream, room.URI)
 	return nil
 }
 
-func (b *Bgitter) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
+func (b *Bgitter) Send(msg config.Message) (string, error) {
+	b.Log.Debugf("=> Receiving %#v", msg)
 	roomID := b.getRoomID(msg.Channel)
 	if roomID == "" {
-		flog.Errorf("Could not find roomID for %v", msg.Channel)
-		return nil
+		b.Log.Errorf("Could not find roomID for %v", msg.Channel)
+		return "", nil
 	}
-	// add ZWSP because gitter echoes our own messages
-	return b.c.SendMessage(roomID, msg.Username+msg.Text+" ​")
+
+	// Delete message
+	if msg.Event == config.EVENT_MSG_DELETE {
+		if msg.ID == "" {
+			return "", nil
+		}
+		// gitter has no delete message api so we edit message to ""
+		_, err := b.c.UpdateMessage(roomID, msg.ID, "")
+		if err != nil {
+			return "", err
+		}
+		return "", nil
+	}
+
+	// Upload a file (in gitter case send the upload URL because gitter has no native upload support)
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.c.SendMessage(roomID, rmsg.Username+rmsg.Text)
+		}
+		if len(msg.Extra["file"]) > 0 {
+			return b.handleUploadFile(&msg, roomID)
+		}
+	}
+
+	// Edit message
+	if msg.ID != "" {
+		b.Log.Debugf("updating message with id %s", msg.ID)
+		_, err := b.c.UpdateMessage(roomID, msg.ID, msg.Username+msg.Text)
+		if err != nil {
+			return "", err
+		}
+		return "", nil
+	}
+
+	// Post normal message
+	resp, err := b.c.SendMessage(roomID, msg.Username+msg.Text)
+	if err != nil {
+		return "", err
+	}
+	return resp.ID, nil
 }
 
 func (b *Bgitter) getRoomID(channel string) string {
@@ -123,3 +160,23 @@ func (b *Bgitter) getAvatar(user string) string {
 	}
 	return avatar
 }
+
+func (b *Bgitter) handleUploadFile(msg *config.Message, roomID string) (string, error) {
+	for _, f := range msg.Extra["file"] {
+		fi := f.(config.FileInfo)
+		if fi.Comment != "" {
+			msg.Text += fi.Comment + ": "
+		}
+		if fi.URL != "" {
+			msg.Text = fi.URL
+			if fi.Comment != "" {
+				msg.Text = fi.Comment + ": " + fi.URL
+			}
+		}
+		_, err := b.c.SendMessage(roomID, msg.Username+msg.Text)
+		if err != nil {
+			return "", err
+		}
+	}
+	return "", nil
+}

bridge/helper/helper.go

@@ -0,0 +1,117 @@
+package helper
+
+import (
+	"bytes"
+	"fmt"
+	"io"
+	"net/http"
+	"regexp"
+	"strings"
+	"time"
+
+	"github.com/42wim/matterbridge/bridge/config"
+	log "github.com/sirupsen/logrus"
+)
+
+func DownloadFile(url string) (*[]byte, error) {
+	return DownloadFileAuth(url, "")
+}
+
+func DownloadFileAuth(url string, auth string) (*[]byte, error) {
+	var buf bytes.Buffer
+	client := &http.Client{
+		Timeout: time.Second * 5,
+	}
+	req, err := http.NewRequest("GET", url, nil)
+	if auth != "" {
+		req.Header.Add("Authorization", auth)
+	}
+	if err != nil {
+		return nil, err
+	}
+	resp, err := client.Do(req)
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+	io.Copy(&buf, resp.Body)
+	data := buf.Bytes()
+	return &data, nil
+}
+
+func SplitStringLength(input string, length int) string {
+	a := []rune(input)
+	str := ""
+	for i, r := range a {
+		str = str + string(r)
+		if i > 0 && (i+1)%length == 0 {
+			str += "\n"
+		}
+	}
+	return str
+}
+
+// handle all the stuff we put into extra
+func HandleExtra(msg *config.Message, general *config.Protocol) []config.Message {
+	extra := msg.Extra
+	rmsg := []config.Message{}
+	if len(extra[config.EVENT_FILE_FAILURE_SIZE]) > 0 {
+		for _, f := range extra[config.EVENT_FILE_FAILURE_SIZE] {
+			fi := f.(config.FileInfo)
+			text := fmt.Sprintf("file %s too big to download (%#v > allowed size: %#v)", fi.Name, fi.Size, general.MediaDownloadSize)
+			rmsg = append(rmsg, config.Message{Text: text, Username: "<system> ", Channel: msg.Channel})
+		}
+		return rmsg
+	}
+	return rmsg
+}
+
+func GetAvatar(av map[string]string, userid string, general *config.Protocol) string {
+	if sha, ok := av[userid]; ok {
+		return general.MediaServerDownload + "/" + sha + "/" + userid + ".png"
+	}
+	return ""
+}
+
+func HandleDownloadSize(flog *log.Entry, msg *config.Message, name string, size int64, general *config.Protocol) error {
+	// check blacklist here
+	for _, entry := range general.MediaDownloadBlackList {
+		if entry != "" {
+			re, err := regexp.Compile(entry)
+			if err != nil {
+				flog.Errorf("incorrect regexp %s for %s", entry, msg.Account)
+				continue
+			}
+			if re.MatchString(name) {
+				return fmt.Errorf("Matching blacklist %s. Not downloading %s", entry, name)
+			}
+		}
+	}
+	flog.Debugf("Trying to download %#v with size %#v", name, size)
+	if int(size) > general.MediaDownloadSize {
+		msg.Event = config.EVENT_FILE_FAILURE_SIZE
+		msg.Extra[msg.Event] = append(msg.Extra[msg.Event], config.FileInfo{Name: name, Comment: msg.Text, Size: size})
+		return fmt.Errorf("File %#v to large to download (%#v). MediaDownloadSize is %#v", name, size, general.MediaDownloadSize)
+	}
+	return nil
+}
+
+func HandleDownloadData(flog *log.Entry, msg *config.Message, name, comment, url string, data *[]byte, general *config.Protocol) {
+	var avatar bool
+	flog.Debugf("Download OK %#v %#v", name, len(*data))
+	if msg.Event == config.EVENT_AVATAR_DOWNLOAD {
+		avatar = true
+	}
+	msg.Extra["file"] = append(msg.Extra["file"], config.FileInfo{Name: name, Data: data, URL: url, Comment: comment, Avatar: avatar})
+}
+
+func RemoveEmptyNewLines(msg string) string {
+	lines := ""
+	for _, line := range strings.Split(msg, "\n") {
+		if line != "" {
+			lines += line + "\n"
+		}
+	}
+	lines = strings.TrimRight(lines, "\n")
+	return lines
+}

bridge/irc/helper.go

@@ -4,6 +4,7 @@ import (
 	"strings"
 )
 
+/*
 func tableformatter(nicks []string, nicksPerRow int, continued bool) string {
 	result := "|IRC users"
 	if continued {
@@ -29,6 +30,7 @@ func tableformatter(nicks []string, nicksPerRow int, continued bool) string {
 	}
 	return result
 }
+*/
 
 func plainformatter(nicks []string, nicksPerRow int) string {
 	return strings.Join(nicks, ", ") + " currently on IRC"

bridge/irc/irc.go

@@ -1,140 +1,271 @@
 package birc
 
 import (
+	"bytes"
 	"crypto/tls"
 	"fmt"
-	"github.com/42wim/matterbridge/bridge/config"
-	log "github.com/Sirupsen/logrus"
-	ircm "github.com/sorcix/irc"
-	"github.com/thoj/go-ircevent"
+	"hash/crc32"
+	"io"
+	"io/ioutil"
+	"net"
 	"regexp"
 	"sort"
 	"strconv"
 	"strings"
 	"time"
+	"unicode/utf8"
+
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
+	"github.com/dfordsoft/golib/ic"
+	"github.com/lrstanley/girc"
+	"github.com/paulrosania/go-charset/charset"
+	_ "github.com/paulrosania/go-charset/data"
+	"github.com/saintfish/chardet"
 )
 
 type Birc struct {
-	i         *irc.Connection
-	Nick      string
-	names     map[string][]string
-	Config    *config.Protocol
-	Remote    chan config.Message
-	connected chan struct{}
-	Local     chan config.Message // local queue for flood control
-	Account   string
+	i                                         *girc.Client
+	Nick                                      string
+	names                                     map[string][]string
+	connected                                 chan struct{}
+	Local                                     chan config.Message // local queue for flood control
+	FirstConnection                           bool
+	MessageDelay, MessageQueue, MessageLength int
+
+	*bridge.Config
 }
 
-var flog *log.Entry
-var protocol = "irc"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Birc {
+func New(cfg *bridge.Config) bridge.Bridger {
 	b := &Birc{}
-	b.Config = &cfg
-	b.Nick = b.Config.Nick
-	b.Remote = c
+	b.Config = cfg
+	b.Nick = b.GetString("Nick")
 	b.names = make(map[string][]string)
-	b.Account = account
 	b.connected = make(chan struct{})
-	if b.Config.MessageDelay == 0 {
-		b.Config.MessageDelay = 1300
+	if b.GetInt("MessageDelay") == 0 {
+		b.MessageDelay = 1300
+	} else {
+		b.MessageDelay = b.GetInt("MessageDelay")
 	}
-	if b.Config.MessageQueue == 0 {
-		b.Config.MessageQueue = 30
+	if b.GetInt("MessageQueue") == 0 {
+		b.MessageQueue = 30
+	} else {
+		b.MessageQueue = b.GetInt("MessageQueue")
 	}
+	if b.GetInt("MessageLength") == 0 {
+		b.MessageLength = 400
+	} else {
+		b.MessageLength = b.GetInt("MessageLength")
+	}
+	b.FirstConnection = true
 	return b
 }
 
 func (b *Birc) Command(msg *config.Message) string {
 	switch msg.Text {
 	case "!users":
-		b.i.AddCallback(ircm.RPL_NAMREPLY, b.storeNames)
-		b.i.AddCallback(ircm.RPL_ENDOFNAMES, b.endNames)
-		b.i.SendRaw("NAMES " + msg.Channel)
+		b.i.Handlers.Add(girc.RPL_NAMREPLY, b.storeNames)
+		b.i.Handlers.Add(girc.RPL_ENDOFNAMES, b.endNames)
+		b.i.Cmd.SendRaw("NAMES " + msg.Channel)
 	}
 	return ""
 }
 
 func (b *Birc) Connect() error {
-	b.Local = make(chan config.Message, b.Config.MessageQueue+10)
-	flog.Infof("Connecting %s", b.Config.Server)
-	i := irc.IRC(b.Config.Nick, b.Config.Nick)
-	if log.GetLevel() == log.DebugLevel {
-		i.Debug = true
-	}
-	i.UseTLS = b.Config.UseTLS
-	i.UseSASL = b.Config.UseSASL
-	i.SASLLogin = b.Config.NickServNick
-	i.SASLPassword = b.Config.NickServPassword
-	i.TLSConfig = &tls.Config{InsecureSkipVerify: b.Config.SkipTLSVerify}
-	if b.Config.Password != "" {
-		i.Password = b.Config.Password
-	}
-	i.AddCallback(ircm.RPL_WELCOME, b.handleNewConnection)
-	err := i.Connect(b.Config.Server)
+	b.Local = make(chan config.Message, b.MessageQueue+10)
+	b.Log.Infof("Connecting %s", b.GetString("Server"))
+	server, portstr, err := net.SplitHostPort(b.GetString("Server"))
 	if err != nil {
 		return err
 	}
+	port, err := strconv.Atoi(portstr)
+	if err != nil {
+		return err
+	}
+	// fix strict user handling of girc
+	user := b.GetString("Nick")
+	for !girc.IsValidUser(user) {
+		if len(user) == 1 {
+			user = "matterbridge"
+			break
+		}
+		user = user[1:]
+	}
+
+	i := girc.New(girc.Config{
+		Server:     server,
+		ServerPass: b.GetString("Password"),
+		Port:       port,
+		Nick:       b.GetString("Nick"),
+		User:       user,
+		Name:       b.GetString("Nick"),
+		SSL:        b.GetBool("UseTLS"),
+		TLSConfig:  &tls.Config{InsecureSkipVerify: b.GetBool("SkipTLSVerify"), ServerName: server},
+		PingDelay:  time.Minute,
+	})
+
+	if b.GetBool("UseSASL") {
+		i.Config.SASL = &girc.SASLPlain{b.GetString("NickServNick"), b.GetString("NickServPassword")}
+	}
+
+	i.Handlers.Add(girc.RPL_WELCOME, b.handleNewConnection)
+	i.Handlers.Add(girc.RPL_ENDOFMOTD, b.handleOtherAuth)
+	i.Handlers.Add(girc.ALL_EVENTS, b.handleOther)
+	go func() {
+		for {
+			if err := i.Connect(); err != nil {
+				b.Log.Errorf("disconnect: error: %s", err)
+			} else {
+				b.Log.Info("disconnect: client requested quit")
+			}
+
+			b.Log.Info("reconnecting in 30 seconds...")
+			time.Sleep(30 * time.Second)
+			i.Handlers.Clear(girc.RPL_WELCOME)
+			i.Handlers.Add(girc.RPL_WELCOME, func(client *girc.Client, event girc.Event) {
+				b.Remote <- config.Message{Username: "system", Text: "rejoin", Channel: "", Account: b.Account, Event: config.EVENT_REJOIN_CHANNELS}
+				// set our correct nick on reconnect if necessary
+				b.Nick = event.Source.Name
+			})
+		}
+	}()
 	b.i = i
 	select {
 	case <-b.connected:
-		flog.Info("Connection succeeded")
+		b.Log.Info("Connection succeeded")
 	case <-time.After(time.Second * 30):
 		return fmt.Errorf("connection timed out")
 	}
-	i.Debug = false
+	//i.Debug = false
+	if b.GetInt("DebugLevel") == 0 {
+		i.Handlers.Clear(girc.ALL_EVENTS)
+	}
 	go b.doSend()
 	return nil
 }
 
 func (b *Birc) Disconnect() error {
-	b.i.Disconnect()
+	b.i.Close()
 	close(b.Local)
 	return nil
 }
 
-func (b *Birc) JoinChannel(channel string) error {
-	b.i.Join(channel)
+func (b *Birc) JoinChannel(channel config.ChannelInfo) error {
+	if channel.Options.Key != "" {
+		b.Log.Debugf("using key %s for channel %s", channel.Options.Key, channel.Name)
+		b.i.Cmd.JoinKey(channel.Name, channel.Options.Key)
+	} else {
+		b.i.Cmd.Join(channel.Name)
+	}
 	return nil
 }
 
-func (b *Birc) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
-	if msg.Account == b.Account {
-		return nil
+func (b *Birc) Send(msg config.Message) (string, error) {
+	// ignore delete messages
+	if msg.Event == config.EVENT_MSG_DELETE {
+		return "", nil
 	}
+
+	b.Log.Debugf("=> Receiving %#v", msg)
+
+	// we can be in between reconnects #385
+	if !b.i.IsConnected() {
+		b.Log.Error("Not connected to server, dropping message")
+	}
+
+	// Execute a command
 	if strings.HasPrefix(msg.Text, "!") {
 		b.Command(&msg)
-		return nil
 	}
-	for _, text := range strings.Split(msg.Text, "\n") {
-		if len(b.Local) < b.Config.MessageQueue {
-			if len(b.Local) == b.Config.MessageQueue-1 {
-				text = text + " <message clipped>"
+
+	// convert to specified charset
+	if b.GetString("Charset") != "" {
+		switch b.GetString("Charset") {
+		case "gbk", "gb18030", "gb2312", "big5", "euc-kr", "euc-jp", "shift-jis", "iso-2022-jp":
+			msg.Text = ic.ConvertString("utf-8", b.GetString("Charset"), msg.Text)
+		default:
+			buf := new(bytes.Buffer)
+			w, err := charset.NewWriter(b.GetString("Charset"), buf)
+			if err != nil {
+				b.Log.Errorf("charset from utf-8 conversion failed: %s", err)
+				return "", err
 			}
-			b.Local <- config.Message{Text: text, Username: msg.Username, Channel: msg.Channel}
-		} else {
-			flog.Debugf("flooding, dropping message (queue at %d)", len(b.Local))
+			fmt.Fprint(w, msg.Text)
+			w.Close()
+			msg.Text = buf.String()
 		}
 	}
-	return nil
+
+	// Handle files
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.Local <- rmsg
+		}
+		if len(msg.Extra["file"]) > 0 {
+			for _, f := range msg.Extra["file"] {
+				fi := f.(config.FileInfo)
+				if fi.Comment != "" {
+					msg.Text += fi.Comment + ": "
+				}
+				if fi.URL != "" {
+					msg.Text = fi.URL
+					if fi.Comment != "" {
+						msg.Text = fi.Comment + ": " + fi.URL
+					}
+				}
+				b.Local <- config.Message{Text: msg.Text, Username: msg.Username, Channel: msg.Channel, Event: msg.Event}
+			}
+			return "", nil
+		}
+	}
+
+	// split long messages on messageLength, to avoid clipped messages #281
+	if b.GetBool("MessageSplit") {
+		msg.Text = helper.SplitStringLength(msg.Text, b.MessageLength)
+	}
+	for _, text := range strings.Split(msg.Text, "\n") {
+		if len(text) > b.MessageLength {
+			text = text[:b.MessageLength-len(" <message clipped>")]
+			if r, size := utf8.DecodeLastRuneInString(text); r == utf8.RuneError {
+				text = text[:len(text)-size]
+			}
+			text += " <message clipped>"
+		}
+		if len(b.Local) < b.MessageQueue {
+			if len(b.Local) == b.MessageQueue-1 {
+				text = text + " <message clipped>"
+			}
+			b.Local <- config.Message{Text: text, Username: msg.Username, Channel: msg.Channel, Event: msg.Event}
+		} else {
+			b.Log.Debugf("flooding, dropping message (queue at %d)", len(b.Local))
+		}
+	}
+	return "", nil
 }
 
 func (b *Birc) doSend() {
-	rate := time.Millisecond * time.Duration(b.Config.MessageDelay)
-	throttle := time.Tick(rate)
+	rate := time.Millisecond * time.Duration(b.MessageDelay)
+	throttle := time.NewTicker(rate)
 	for msg := range b.Local {
-		<-throttle
-		b.i.Privmsg(msg.Channel, msg.Username+msg.Text)
+		<-throttle.C
+		username := msg.Username
+		if b.GetBool("Colornicks") {
+			checksum := crc32.ChecksumIEEE([]byte(msg.Username))
+			colorCode := checksum%14 + 2 // quick fix - prevent white or black color codes
+			username = fmt.Sprintf("\x03%02d%s\x0F", colorCode, msg.Username)
+		}
+		if msg.Event == config.EVENT_USER_ACTION {
+			b.i.Cmd.Action(msg.Channel, username+msg.Text)
+		} else {
+			b.Log.Debugf("Sending to channel %s", msg.Channel)
+			b.i.Cmd.Message(msg.Channel, username+msg.Text)
+		}
 	}
 }
 
-func (b *Birc) endNames(event *irc.Event) {
-	channel := event.Arguments[1]
+func (b *Birc) endNames(client *girc.Client, event girc.Event) {
+	channel := event.Params[1]
 	sort.Strings(b.names[channel])
 	maxNamesPerPost := (300 / b.nicksPerRow()) * b.nicksPerRow()
 	continued := false
@@ -147,121 +278,188 @@ func (b *Birc) endNames(event *irc.Event) {
 	b.Remote <- config.Message{Username: b.Nick, Text: b.formatnicks(b.names[channel], continued),
 		Channel: channel, Account: b.Account}
 	b.names[channel] = nil
-	b.i.ClearCallback(ircm.RPL_NAMREPLY)
-	b.i.ClearCallback(ircm.RPL_ENDOFNAMES)
+	b.i.Handlers.Clear(girc.RPL_NAMREPLY)
+	b.i.Handlers.Clear(girc.RPL_ENDOFNAMES)
 }
 
-func (b *Birc) handleNewConnection(event *irc.Event) {
-	flog.Debug("Registering callbacks")
+func (b *Birc) handleNewConnection(client *girc.Client, event girc.Event) {
+	b.Log.Debug("Registering callbacks")
 	i := b.i
-	b.Nick = event.Arguments[0]
-	i.AddCallback("PRIVMSG", b.handlePrivMsg)
-	i.AddCallback("CTCP_ACTION", b.handlePrivMsg)
-	i.AddCallback(ircm.RPL_TOPICWHOTIME, b.handleTopicWhoTime)
-	i.AddCallback(ircm.NOTICE, b.handleNotice)
-	//i.AddCallback(ircm.RPL_MYINFO, func(e *irc.Event) { flog.Infof("%s: %s", e.Code, strings.Join(e.Arguments[1:], " ")) })
-	i.AddCallback("PING", func(e *irc.Event) {
-		i.SendRaw("PONG :" + e.Message())
-		flog.Debugf("PING/PONG")
-	})
-	i.AddCallback("JOIN", b.handleJoinPart)
-	i.AddCallback("PART", b.handleJoinPart)
-	i.AddCallback("QUIT", b.handleJoinPart)
-	i.AddCallback("*", b.handleOther)
+	b.Nick = event.Params[0]
+
+	i.Handlers.Add(girc.RPL_ENDOFMOTD, b.handleOtherAuth)
+	i.Handlers.Add("PRIVMSG", b.handlePrivMsg)
+	i.Handlers.Add("CTCP_ACTION", b.handlePrivMsg)
+	i.Handlers.Add(girc.RPL_TOPICWHOTIME, b.handleTopicWhoTime)
+	i.Handlers.Add(girc.NOTICE, b.handleNotice)
+	i.Handlers.Add("JOIN", b.handleJoinPart)
+	i.Handlers.Add("PART", b.handleJoinPart)
+	i.Handlers.Add("QUIT", b.handleJoinPart)
+	i.Handlers.Add("KICK", b.handleJoinPart)
 	// we are now fully connected
 	b.connected <- struct{}{}
 }
 
-func (b *Birc) handleJoinPart(event *irc.Event) {
-	flog.Debugf("Sending JOIN_LEAVE event from %s to gateway", b.Account)
-	channel := event.Arguments[0]
-	if event.Code == "QUIT" {
-		if event.Nick == b.Nick && strings.Contains(event.Raw, "Ping timeout") {
-			flog.Infof("%s reconnecting ..", b.Account)
+func (b *Birc) handleJoinPart(client *girc.Client, event girc.Event) {
+	if len(event.Params) == 0 {
+		b.Log.Debugf("handleJoinPart: empty Params? %#v", event)
+		return
+	}
+	channel := strings.ToLower(event.Params[0])
+	if event.Command == "KICK" {
+		b.Log.Infof("Got kicked from %s by %s", channel, event.Source.Name)
+		time.Sleep(time.Duration(b.GetInt("RejoinDelay")) * time.Second)
+		b.Remote <- config.Message{Username: "system", Text: "rejoin", Channel: channel, Account: b.Account, Event: config.EVENT_REJOIN_CHANNELS}
+		return
+	}
+	if event.Command == "QUIT" {
+		if event.Source.Name == b.Nick && strings.Contains(event.Trailing, "Ping timeout") {
+			b.Log.Infof("%s reconnecting ..", b.Account)
 			b.Remote <- config.Message{Username: "system", Text: "reconnect", Channel: channel, Account: b.Account, Event: config.EVENT_FAILURE}
 			return
 		}
 	}
-	b.Remote <- config.Message{Username: "system", Text: event.Nick + " " + strings.ToLower(event.Code) + "s", Channel: channel, Account: b.Account, Event: config.EVENT_JOIN_LEAVE}
-	flog.Debugf("handle %#v", event)
+	if event.Source.Name != b.Nick {
+		if b.GetBool("nosendjoinpart") {
+			return
+		}
+		b.Log.Debugf("<= Sending JOIN_LEAVE event from %s to gateway", b.Account)
+		msg := config.Message{Username: "system", Text: event.Source.Name + " " + strings.ToLower(event.Command) + "s", Channel: channel, Account: b.Account, Event: config.EVENT_JOIN_LEAVE}
+		b.Log.Debugf("<= Message is %#v", msg)
+		b.Remote <- msg
+		return
+	}
+	b.Log.Debugf("handle %#v", event)
 }
 
-func (b *Birc) handleNotice(event *irc.Event) {
-	if strings.Contains(event.Message(), "This nickname is registered") && event.Nick == b.Config.NickServNick {
-		b.i.Privmsg(b.Config.NickServNick, "IDENTIFY "+b.Config.NickServPassword)
+func (b *Birc) handleNotice(client *girc.Client, event girc.Event) {
+	if strings.Contains(event.String(), "This nickname is registered") && event.Source.Name == b.GetString("NickServNick") {
+		b.i.Cmd.Message(b.GetString("NickServNick"), "IDENTIFY "+b.GetString("NickServPassword"))
 	} else {
-		b.handlePrivMsg(event)
+		b.handlePrivMsg(client, event)
 	}
 }
 
-func (b *Birc) handleOther(event *irc.Event) {
-	switch event.Code {
+func (b *Birc) handleOther(client *girc.Client, event girc.Event) {
+	if b.GetInt("DebugLevel") == 1 {
+		if event.Command != "CLIENT_STATE_UPDATED" &&
+			event.Command != "CLIENT_GENERAL_UPDATED" {
+			b.Log.Debugf("%#v", event.String())
+		}
+		return
+	}
+	switch event.Command {
 	case "372", "375", "376", "250", "251", "252", "253", "254", "255", "265", "266", "002", "003", "004", "005":
 		return
 	}
-	flog.Debugf("%#v", event.Raw)
+	b.Log.Debugf("%#v", event.String())
 }
 
-func (b *Birc) handlePrivMsg(event *irc.Event) {
+func (b *Birc) handleOtherAuth(client *girc.Client, event girc.Event) {
+	if strings.EqualFold(b.GetString("NickServNick"), "Q@CServe.quakenet.org") {
+		b.Log.Debugf("Authenticating %s against %s", b.GetString("NickServUsername"), b.GetString("NickServNick"))
+		b.i.Cmd.Message(b.GetString("NickServNick"), "AUTH "+b.GetString("NickServUsername")+" "+b.GetString("NickServPassword"))
+	}
+}
+
+func (b *Birc) skipPrivMsg(event girc.Event) bool {
+	// Our nick can be changed
+	b.Nick = b.i.GetNick()
+
+	// freenode doesn't send 001 as first reply
+	if event.Command == "NOTICE" {
+		return true
+	}
 	// don't forward queries to the bot
-	if event.Arguments[0] == b.Nick {
-		return
+	if event.Params[0] == b.Nick {
+		return true
 	}
 	// don't forward message from ourself
-	if event.Nick == b.Nick {
-		return
+	if event.Source.Name == b.Nick {
+		return true
 	}
-	flog.Debugf("handlePrivMsg() %s %s %#v", event.Nick, event.Message(), event)
-	msg := ""
-	if event.Code == "CTCP_ACTION" {
-		msg = event.Nick + " "
-	}
-	msg += event.Message()
-	// strip IRC colors
-	re := regexp.MustCompile(`[[:cntrl:]](\d+,|)\d+`)
-	msg = re.ReplaceAllString(msg, "")
-	flog.Debugf("Sending message from %s on %s to gateway", event.Arguments[0], b.Account)
-	b.Remote <- config.Message{Username: event.Nick, Text: msg, Channel: event.Arguments[0], Account: b.Account}
+	return false
 }
 
-func (b *Birc) handleTopicWhoTime(event *irc.Event) {
-	parts := strings.Split(event.Arguments[2], "!")
-	t, err := strconv.ParseInt(event.Arguments[3], 10, 64)
+func (b *Birc) handlePrivMsg(client *girc.Client, event girc.Event) {
+	if b.skipPrivMsg(event) {
+		return
+	}
+	rmsg := config.Message{Username: event.Source.Name, Channel: strings.ToLower(event.Params[0]), Account: b.Account, UserID: event.Source.Ident + "@" + event.Source.Host}
+	b.Log.Debugf("== Receiving PRIVMSG: %s %s %#v", event.Source.Name, event.Trailing, event)
+
+	// set action event
+	if event.IsAction() {
+		rmsg.Event = config.EVENT_USER_ACTION
+	}
+
+	// strip action, we made an event if it was an action
+	rmsg.Text += event.StripAction()
+
+	// strip IRC colors
+	re := regexp.MustCompile(`[[:cntrl:]](?:\d{1,2}(?:,\d{1,2})?)?`)
+	rmsg.Text = re.ReplaceAllString(rmsg.Text, "")
+
+	// start detecting the charset
+	var r io.Reader
+	var err error
+	mycharset := b.GetString("Charset")
+	if mycharset == "" {
+		// detect what were sending so that we convert it to utf-8
+		detector := chardet.NewTextDetector()
+		result, err := detector.DetectBest([]byte(rmsg.Text))
+		if err != nil {
+			b.Log.Infof("detection failed for rmsg.Text: %#v", rmsg.Text)
+			return
+		}
+		b.Log.Debugf("detected %s confidence %#v", result.Charset, result.Confidence)
+		mycharset = result.Charset
+		// if we're not sure, just pick ISO-8859-1
+		if result.Confidence < 80 {
+			mycharset = "ISO-8859-1"
+		}
+	}
+	switch mycharset {
+	case "gbk", "gb18030", "gb2312", "big5", "euc-kr", "euc-jp", "shift-jis", "iso-2022-jp":
+		rmsg.Text = ic.ConvertString("utf-8", b.GetString("Charset"), rmsg.Text)
+	default:
+		r, err = charset.NewReader(mycharset, strings.NewReader(rmsg.Text))
+		if err != nil {
+			b.Log.Errorf("charset to utf-8 conversion failed: %s", err)
+			return
+		}
+		output, _ := ioutil.ReadAll(r)
+		rmsg.Text = string(output)
+	}
+
+	b.Log.Debugf("<= Sending message from %s on %s to gateway", event.Params[0], b.Account)
+	b.Remote <- rmsg
+}
+
+func (b *Birc) handleTopicWhoTime(client *girc.Client, event girc.Event) {
+	parts := strings.Split(event.Params[2], "!")
+	t, err := strconv.ParseInt(event.Params[3], 10, 64)
 	if err != nil {
-		flog.Errorf("Invalid time stamp: %s", event.Arguments[3])
+		b.Log.Errorf("Invalid time stamp: %s", event.Params[3])
 	}
 	user := parts[0]
 	if len(parts) > 1 {
 		user += " [" + parts[1] + "]"
 	}
-	flog.Debugf("%s: Topic set by %s [%s]", event.Code, user, time.Unix(t, 0))
+	b.Log.Debugf("%s: Topic set by %s [%s]", event.Command, user, time.Unix(t, 0))
 }
 
 func (b *Birc) nicksPerRow() int {
 	return 4
-	/*
-		if b.Config.Mattermost.NicksPerRow < 1 {
-			return 4
-		}
-		return b.Config.Mattermost.NicksPerRow
-	*/
 }
 
-func (b *Birc) storeNames(event *irc.Event) {
-	channel := event.Arguments[2]
+func (b *Birc) storeNames(client *girc.Client, event girc.Event) {
+	channel := event.Params[2]
 	b.names[channel] = append(
 		b.names[channel],
-		strings.Split(strings.TrimSpace(event.Message()), " ")...)
+		strings.Split(strings.TrimSpace(event.Trailing), " ")...)
 }
 
 func (b *Birc) formatnicks(nicks []string, continued bool) string {
 	return plainformatter(nicks, b.nicksPerRow())
-	/*
-		switch b.Config.Mattermost.NickFormatter {
-		case "table":
-			return tableformatter(nicks, b.nicksPerRow(), continued)
-		default:
-			return plainformatter(nicks, b.nicksPerRow())
-		}
-	*/
 }

bridge/matrix/matrix.go

@@ -1,58 +1,51 @@
 package bmatrix
 
 import (
-	"github.com/42wim/matterbridge/bridge/config"
-	log "github.com/Sirupsen/logrus"
-	matrix "github.com/matrix-org/gomatrix"
+	"bytes"
+	"fmt"
+	"mime"
+	"regexp"
+	"strings"
 	"sync"
+
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
+	matrix "github.com/matterbridge/gomatrix"
 )
 
 type Bmatrix struct {
 	mc      *matrix.Client
-	Config  *config.Protocol
-	Remote  chan config.Message
-	Account string
 	UserID  string
 	RoomMap map[string]string
 	sync.RWMutex
+	*bridge.Config
 }
 
-var flog *log.Entry
-var protocol = "matrix"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Bmatrix {
-	b := &Bmatrix{}
+func New(cfg *bridge.Config) bridge.Bridger {
+	b := &Bmatrix{Config: cfg}
 	b.RoomMap = make(map[string]string)
-	b.Config = &cfg
-	b.Account = account
-	b.Remote = c
 	return b
 }
 
 func (b *Bmatrix) Connect() error {
 	var err error
-	flog.Infof("Connecting %s", b.Config.Server)
-	b.mc, err = matrix.NewClient(b.Config.Server, "", "")
+	b.Log.Infof("Connecting %s", b.GetString("Server"))
+	b.mc, err = matrix.NewClient(b.GetString("Server"), "", "")
 	if err != nil {
-		flog.Debugf("%#v", err)
 		return err
 	}
 	resp, err := b.mc.Login(&matrix.ReqLogin{
 		Type:     "m.login.password",
-		User:     b.Config.Login,
-		Password: b.Config.Password,
+		User:     b.GetString("Login"),
+		Password: b.GetString("Password"),
 	})
 	if err != nil {
-		flog.Debugf("%#v", err)
 		return err
 	}
 	b.mc.SetCredentials(resp.UserID, resp.AccessToken)
 	b.UserID = resp.UserID
-	flog.Info("Connection succeeded")
+	b.Log.Info("Connection succeeded")
 	go b.handlematrix()
 	return nil
 }
@@ -61,23 +54,65 @@ func (b *Bmatrix) Disconnect() error {
 	return nil
 }
 
-func (b *Bmatrix) JoinChannel(channel string) error {
-	resp, err := b.mc.JoinRoom(channel, "", nil)
+func (b *Bmatrix) JoinChannel(channel config.ChannelInfo) error {
+	resp, err := b.mc.JoinRoom(channel.Name, "", nil)
 	if err != nil {
 		return err
 	}
 	b.Lock()
-	b.RoomMap[resp.RoomID] = channel
+	b.RoomMap[resp.RoomID] = channel.Name
 	b.Unlock()
 	return err
 }
 
-func (b *Bmatrix) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
+func (b *Bmatrix) Send(msg config.Message) (string, error) {
+	b.Log.Debugf("=> Receiving %#v", msg)
+
 	channel := b.getRoomID(msg.Channel)
-	flog.Debugf("Sending to channel %s", channel)
-	b.mc.SendText(channel, msg.Username+msg.Text)
-	return nil
+	b.Log.Debugf("Channel %s maps to channel id %s", msg.Channel, channel)
+
+	// Make a action /me of the message
+	if msg.Event == config.EVENT_USER_ACTION {
+		resp, err := b.mc.SendMessageEvent(channel, "m.room.message",
+			matrix.TextMessage{"m.emote", msg.Username + msg.Text})
+		if err != nil {
+			return "", err
+		}
+		return resp.EventID, err
+	}
+
+	// Delete message
+	if msg.Event == config.EVENT_MSG_DELETE {
+		if msg.ID == "" {
+			return "", nil
+		}
+		resp, err := b.mc.RedactEvent(channel, msg.ID, &matrix.ReqRedact{})
+		if err != nil {
+			return "", err
+		}
+		return resp.EventID, err
+	}
+
+	// Upload a file if it exists
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.mc.SendText(channel, rmsg.Username+rmsg.Text)
+		}
+		// check if we have files to upload (from slack, telegram or mattermost)
+		if len(msg.Extra["file"]) > 0 {
+			return b.handleUploadFile(&msg, channel)
+		}
+	}
+
+	// Edit message if we have an ID
+	// matrix has no editing support
+
+	// Post normal message
+	resp, err := b.mc.SendText(channel, msg.Username+msg.Text)
+	if err != nil {
+		return "", err
+	}
+	return resp.EventID, err
 }
 
 func (b *Bmatrix) getRoomID(channel string) string {
@@ -90,28 +125,188 @@ func (b *Bmatrix) getRoomID(channel string) string {
 	}
 	return ""
 }
+
 func (b *Bmatrix) handlematrix() error {
 	syncer := b.mc.Syncer.(*matrix.DefaultSyncer)
-	syncer.OnEventType("m.room.message", func(ev *matrix.Event) {
-		if ev.Content["msgtype"].(string) == "m.text" && ev.Sender != b.UserID {
-			b.RLock()
-			channel, ok := b.RoomMap[ev.RoomID]
-			b.RUnlock()
-			if !ok {
-				flog.Debugf("Unknown room %s", ev.RoomID)
-				return
-			}
-			flog.Debugf("Sending message from %s on %s to gateway", ev.Sender, b.Account)
-			b.Remote <- config.Message{Username: ev.Sender, Text: ev.Content["body"].(string), Channel: channel, Account: b.Account}
-		}
-		flog.Debugf("Received: %#v", ev)
-	})
+	syncer.OnEventType("m.room.redaction", b.handleEvent)
+	syncer.OnEventType("m.room.message", b.handleEvent)
 	go func() {
 		for {
 			if err := b.mc.Sync(); err != nil {
-				flog.Println("Sync() returned ", err)
+				b.Log.Println("Sync() returned ", err)
 			}
 		}
 	}()
 	return nil
 }
+
+func (b *Bmatrix) handleEvent(ev *matrix.Event) {
+	b.Log.Debugf("== Receiving event: %#v", ev)
+	if ev.Sender != b.UserID {
+		b.RLock()
+		channel, ok := b.RoomMap[ev.RoomID]
+		b.RUnlock()
+		if !ok {
+			b.Log.Debugf("Unknown room %s", ev.RoomID)
+			return
+		}
+
+		// TODO download avatar
+
+		// Create our message
+		rmsg := config.Message{Username: ev.Sender[1:], Channel: channel, Account: b.Account, UserID: ev.Sender, ID: ev.ID}
+
+		// Text must be a string
+		if rmsg.Text, ok = ev.Content["body"].(string); !ok {
+			b.Log.Errorf("Content[body] wasn't a %T ?", rmsg.Text)
+			return
+		}
+
+		// Remove homeserver suffix if configured
+		if b.GetBool("NoHomeServerSuffix") {
+			re := regexp.MustCompile("(.*?):.*")
+			rmsg.Username = re.ReplaceAllString(rmsg.Username, `$1`)
+		}
+
+		// Delete event
+		if ev.Type == "m.room.redaction" {
+			rmsg.Event = config.EVENT_MSG_DELETE
+			rmsg.ID = ev.Redacts
+			rmsg.Text = config.EVENT_MSG_DELETE
+			b.Remote <- rmsg
+			return
+		}
+
+		// Do we have a /me action
+		if ev.Content["msgtype"].(string) == "m.emote" {
+			rmsg.Event = config.EVENT_USER_ACTION
+		}
+
+		// Do we have attachments
+		if b.containsAttachment(ev.Content) {
+			err := b.handleDownloadFile(&rmsg, ev.Content)
+			if err != nil {
+				b.Log.Errorf("download failed: %#v", err)
+			}
+		}
+
+		b.Log.Debugf("<= Sending message from %s on %s to gateway", ev.Sender, b.Account)
+		b.Remote <- rmsg
+	}
+}
+
+// handleDownloadFile handles file download
+func (b *Bmatrix) handleDownloadFile(rmsg *config.Message, content map[string]interface{}) error {
+	var (
+		ok                        bool
+		url, name, msgtype, mtype string
+		info                      map[string]interface{}
+		size                      float64
+	)
+
+	rmsg.Extra = make(map[string][]interface{})
+	if url, ok = content["url"].(string); !ok {
+		return fmt.Errorf("url isn't a %T", url)
+	}
+	url = strings.Replace(url, "mxc://", b.GetString("Server")+"/_matrix/media/v1/download/", -1)
+
+	if info, ok = content["info"].(map[string]interface{}); !ok {
+		return fmt.Errorf("info isn't a %T", info)
+	}
+	if size, ok = info["size"].(float64); !ok {
+		return fmt.Errorf("size isn't a %T", size)
+	}
+	if name, ok = content["body"].(string); !ok {
+		return fmt.Errorf("name isn't a %T", name)
+	}
+	if msgtype, ok = content["msgtype"].(string); !ok {
+		return fmt.Errorf("msgtype isn't a %T", msgtype)
+	}
+	if mtype, ok = info["mimetype"].(string); !ok {
+		return fmt.Errorf("mtype isn't a %T", mtype)
+	}
+
+	// check if we have an image uploaded without extension
+	if !strings.Contains(name, ".") {
+		if msgtype == "m.image" {
+			mext, _ := mime.ExtensionsByType(mtype)
+			if len(mext) > 0 {
+				name = name + mext[0]
+			}
+		} else {
+			// just a default .png extension if we don't have mime info
+			name = name + ".png"
+		}
+	}
+
+	// check if the size is ok
+	err := helper.HandleDownloadSize(b.Log, rmsg, name, int64(size), b.General)
+	if err != nil {
+		return err
+	}
+	// actually download the file
+	data, err := helper.DownloadFile(url)
+	if err != nil {
+		return fmt.Errorf("download %s failed %#v", url, err)
+	}
+	// add the downloaded data to the message
+	helper.HandleDownloadData(b.Log, rmsg, name, "", url, data, b.General)
+	return nil
+}
+
+// handleUploadFile handles native upload of files
+func (b *Bmatrix) handleUploadFile(msg *config.Message, channel string) (string, error) {
+	for _, f := range msg.Extra["file"] {
+		fi := f.(config.FileInfo)
+		content := bytes.NewReader(*fi.Data)
+		sp := strings.Split(fi.Name, ".")
+		mtype := mime.TypeByExtension("." + sp[len(sp)-1])
+		if strings.Contains(mtype, "image") ||
+			strings.Contains(mtype, "video") {
+			if fi.Comment != "" {
+				_, err := b.mc.SendText(channel, msg.Username+fi.Comment)
+				if err != nil {
+					b.Log.Errorf("file comment failed: %#v", err)
+				}
+			}
+			b.Log.Debugf("uploading file: %s %s", fi.Name, mtype)
+			res, err := b.mc.UploadToContentRepo(content, mtype, int64(len(*fi.Data)))
+			if err != nil {
+				b.Log.Errorf("file upload failed: %#v", err)
+				continue
+			}
+			if strings.Contains(mtype, "video") {
+				b.Log.Debugf("sendVideo %s", res.ContentURI)
+				_, err = b.mc.SendVideo(channel, fi.Name, res.ContentURI)
+				if err != nil {
+					b.Log.Errorf("sendVideo failed: %#v", err)
+				}
+			}
+			if strings.Contains(mtype, "image") {
+				b.Log.Debugf("sendImage %s", res.ContentURI)
+				_, err = b.mc.SendImage(channel, fi.Name, res.ContentURI)
+				if err != nil {
+					b.Log.Errorf("sendImage failed: %#v", err)
+				}
+			}
+			b.Log.Debugf("result: %#v", res)
+		}
+	}
+	return "", nil
+}
+
+// skipMessages returns true if this message should not be handled
+func (b *Bmatrix) containsAttachment(content map[string]interface{}) bool {
+	// Skip empty messages
+	if content["msgtype"] == nil {
+		return false
+	}
+
+	// Only allow image,video or file msgtypes
+	if !(content["msgtype"].(string) == "m.image" ||
+		content["msgtype"].(string) == "m.video" ||
+		content["msgtype"].(string) == "m.file") {
+		return false
+	}
+	return true
+}

bridge/mattermost/mattermost.go

@@ -1,51 +1,30 @@
 package bmattermost
 
 import (
+	"errors"
+	"fmt"
+	"strings"
+
+	"github.com/42wim/matterbridge/bridge"
 	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
 	"github.com/42wim/matterbridge/matterclient"
 	"github.com/42wim/matterbridge/matterhook"
-	log "github.com/Sirupsen/logrus"
+	"github.com/rs/xid"
 )
 
-type MMhook struct {
-	mh *matterhook.Client
-}
-
-type MMapi struct {
-	mc            *matterclient.MMClient
-	mmMap         map[string]string
-	mmIgnoreNicks []string
-}
-
-type MMMessage struct {
-	Text     string
-	Channel  string
-	Username string
-}
-
 type Bmattermost struct {
-	MMhook
-	MMapi
-	Config  *config.Protocol
-	Remote  chan config.Message
-	name    string
-	TeamId  string
-	Account string
+	mh     *matterhook.Client
+	mc     *matterclient.MMClient
+	uuid   string
+	TeamID string
+	*bridge.Config
+	avatarMap map[string]string
 }
 
-var flog *log.Entry
-var protocol = "mattermost"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Bmattermost {
-	b := &Bmattermost{}
-	b.Config = &cfg
-	b.Remote = c
-	b.Account = account
-	b.mmMap = make(map[string]string)
+func New(cfg *bridge.Config) bridge.Bridger {
+	b := &Bmattermost{Config: cfg, avatarMap: make(map[string]string)}
+	b.uuid = xid.New().String()
 	return b
 }
 
@@ -54,26 +33,72 @@ func (b *Bmattermost) Command(cmd string) string {
 }
 
 func (b *Bmattermost) Connect() error {
-	if !b.Config.UseAPI {
-		flog.Info("Connecting webhooks")
-		b.mh = matterhook.New(b.Config.URL,
-			matterhook.Config{InsecureSkipVerify: b.Config.SkipTLSVerify,
-				BindAddress: b.Config.BindAddress})
-	} else {
-		b.mc = matterclient.New(b.Config.Login, b.Config.Password,
-			b.Config.Team, b.Config.Server)
-		b.mc.SkipTLSVerify = b.Config.SkipTLSVerify
-		b.mc.NoTLS = b.Config.NoTLS
-		flog.Infof("Connecting %s (team: %s) on %s", b.Config.Login, b.Config.Team, b.Config.Server)
-		err := b.mc.Login()
+	if b.GetString("WebhookBindAddress") != "" {
+		if b.GetString("WebhookURL") != "" {
+			b.Log.Info("Connecting using webhookurl (sending) and webhookbindaddress (receiving)")
+			b.mh = matterhook.New(b.GetString("WebhookURL"),
+				matterhook.Config{InsecureSkipVerify: b.GetBool("SkipTLSVerify"),
+					BindAddress: b.GetString("WebhookBindAddress")})
+		} else if b.GetString("Token") != "" {
+			b.Log.Info("Connecting using token (sending)")
+			err := b.apiLogin()
+			if err != nil {
+				return err
+			}
+		} else if b.GetString("Login") != "" {
+			b.Log.Info("Connecting using login/password (sending)")
+			err := b.apiLogin()
+			if err != nil {
+				return err
+			}
+		} else {
+			b.Log.Info("Connecting using webhookbindaddress (receiving)")
+			b.mh = matterhook.New(b.GetString("WebhookURL"),
+				matterhook.Config{InsecureSkipVerify: b.GetBool("SkipTLSVerify"),
+					BindAddress: b.GetString("WebhookBindAddress")})
+		}
+		go b.handleMatter()
+		return nil
+	}
+	if b.GetString("WebhookURL") != "" {
+		b.Log.Info("Connecting using webhookurl (sending)")
+		b.mh = matterhook.New(b.GetString("WebhookURL"),
+			matterhook.Config{InsecureSkipVerify: b.GetBool("SkipTLSVerify"),
+				DisableServer: true})
+		if b.GetString("Token") != "" {
+			b.Log.Info("Connecting using token (receiving)")
+			err := b.apiLogin()
+			if err != nil {
+				return err
+			}
+			go b.handleMatter()
+		} else if b.GetString("Login") != "" {
+			b.Log.Info("Connecting using login/password (receiving)")
+			err := b.apiLogin()
+			if err != nil {
+				return err
+			}
+			go b.handleMatter()
+		}
+		return nil
+	} else if b.GetString("Token") != "" {
+		b.Log.Info("Connecting using token (sending and receiving)")
+		err := b.apiLogin()
 		if err != nil {
 			return err
 		}
-		flog.Info("Connection succeeded")
-		b.TeamId = b.mc.GetTeamId()
-		go b.mc.WsReceiver()
+		go b.handleMatter()
+	} else if b.GetString("Login") != "" {
+		b.Log.Info("Connecting using login/password (sending and receiving)")
+		err := b.apiLogin()
+		if err != nil {
+			return err
+		}
+		go b.handleMatter()
+	}
+	if b.GetString("WebhookBindAddress") == "" && b.GetString("WebhookURL") == "" && b.GetString("Login") == "" && b.GetString("Token") == "" {
+		return errors.New("no connection method found. See that you have WebhookBindAddress, WebhookURL or Token/Login/Password/Server/Team configured")
 	}
-	go b.handleMatter()
 	return nil
 }
 
@@ -81,87 +106,357 @@ func (b *Bmattermost) Disconnect() error {
 	return nil
 }
 
-func (b *Bmattermost) JoinChannel(channel string) error {
+func (b *Bmattermost) JoinChannel(channel config.ChannelInfo) error {
 	// we can only join channels using the API
-	if b.Config.UseAPI {
-		return b.mc.JoinChannel(b.mc.GetChannelId(channel, ""))
+	if b.GetString("WebhookURL") == "" && b.GetString("WebhookBindAddress") == "" {
+		id := b.mc.GetChannelId(channel.Name, "")
+		if id == "" {
+			return fmt.Errorf("Could not find channel ID for channel %s", channel.Name)
+		}
+		return b.mc.JoinChannel(id)
 	}
 	return nil
 }
 
-func (b *Bmattermost) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
-	nick := msg.Username
-	message := msg.Text
-	channel := msg.Channel
+func (b *Bmattermost) Send(msg config.Message) (string, error) {
+	b.Log.Debugf("=> Receiving %#v", msg)
 
-	if b.Config.PrefixMessagesWithNick {
-		/*if IsMarkup(message) {
-			message = nick + "\n\n" + message
-		} else {
-		*/
-		message = nick + " " + message
-		//}
+	// Make a action /me of the message
+	if msg.Event == config.EVENT_USER_ACTION {
+		msg.Text = "*" + msg.Text + "*"
 	}
-	if !b.Config.UseAPI {
-		matterMessage := matterhook.OMessage{IconURL: b.Config.IconURL}
-		matterMessage.Channel = channel
-		matterMessage.UserName = nick
-		matterMessage.Type = ""
-		matterMessage.Text = message
-		err := b.mh.Send(matterMessage)
-		if err != nil {
-			flog.Info(err)
-			return err
+
+	// map the file SHA to our user (caches the avatar)
+	if msg.Event == config.EVENT_AVATAR_DOWNLOAD {
+		return b.cacheAvatar(&msg)
+	}
+
+	// Use webhook to send the message
+	if b.GetString("WebhookURL") != "" {
+		return b.sendWebhook(msg)
+	}
+
+	// Delete message
+	if msg.Event == config.EVENT_MSG_DELETE {
+		if msg.ID == "" {
+			return "", nil
 		}
-		return nil
+		return msg.ID, b.mc.DeleteMessage(msg.ID)
 	}
-	b.mc.PostMessage(b.mc.GetChannelId(channel, ""), message)
-	return nil
+
+	// Upload a file if it exists
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.mc.PostMessage(b.mc.GetChannelId(rmsg.Channel, ""), rmsg.Username+rmsg.Text)
+		}
+		if len(msg.Extra["file"]) > 0 {
+			return b.handleUploadFile(&msg)
+		}
+	}
+
+	// Prepend nick if configured
+	if b.GetBool("PrefixMessagesWithNick") {
+		msg.Text = msg.Username + msg.Text
+	}
+
+	// Edit message if we have an ID
+	if msg.ID != "" {
+		return b.mc.EditMessage(msg.ID, msg.Text)
+	}
+
+	// Post normal message
+	return b.mc.PostMessage(b.mc.GetChannelId(msg.Channel, ""), msg.Text)
 }
 
 func (b *Bmattermost) handleMatter() {
-	flog.Debugf("Choosing API based Mattermost connection: %t", b.Config.UseAPI)
-	mchan := make(chan *MMMessage)
-	if b.Config.UseAPI {
-		go b.handleMatterClient(mchan)
+	messages := make(chan *config.Message)
+	if b.GetString("WebhookBindAddress") != "" {
+		b.Log.Debugf("Choosing webhooks based receiving")
+		go b.handleMatterHook(messages)
 	} else {
-		go b.handleMatterHook(mchan)
+		if b.GetString("Token") != "" {
+			b.Log.Debugf("Choosing token based receiving")
+		} else {
+			b.Log.Debugf("Choosing login/password based receiving")
+		}
+		go b.handleMatterClient(messages)
 	}
-	for message := range mchan {
-		flog.Debugf("Sending message from %s on %s to gateway", message.Username, b.Account)
-		b.Remote <- config.Message{Text: message.Text, Username: message.Username, Channel: message.Channel, Account: b.Account}
+	var ok bool
+	for message := range messages {
+		message.Avatar = helper.GetAvatar(b.avatarMap, message.UserID, b.General)
+		message.Account = b.Account
+		message.Text, ok = b.replaceAction(message.Text)
+		if ok {
+			message.Event = config.EVENT_USER_ACTION
+		}
+		b.Log.Debugf("<= Sending message from %s on %s to gateway", message.Username, b.Account)
+		b.Log.Debugf("<= Message is %#v", message)
+		b.Remote <- *message
 	}
 }
 
-func (b *Bmattermost) handleMatterClient(mchan chan *MMMessage) {
+func (b *Bmattermost) handleMatterClient(messages chan *config.Message) {
 	for message := range b.mc.MessageChan {
-		// do not post our own messages back to irc
-		// only listen to message from our team
-		if message.Raw.Event == "posted" && b.mc.User.Username != message.Username && message.Raw.Data["team_id"].(string) == b.TeamId {
-			flog.Debugf("Receiving from matterclient %#v", message)
-			m := &MMMessage{}
-			m.Username = message.Username
-			m.Channel = message.Channel
-			m.Text = message.Text
-			if len(message.Post.FileIds) > 0 {
-				for _, link := range b.mc.GetPublicLinks(message.Post.FileIds) {
-					m.Text = m.Text + "\n" + link
+		b.Log.Debugf("%#v", message.Raw.Data)
+
+		if b.skipMessage(message) {
+			b.Log.Debugf("Skipped message: %#v", message)
+			continue
+		}
+
+		// only download avatars if we have a place to upload them (configured mediaserver)
+		if b.General.MediaServerUpload != "" {
+			b.handleDownloadAvatar(message.UserID, message.Channel)
+		}
+
+		b.Log.Debugf("== Receiving event %#v", message)
+
+		rmsg := &config.Message{Username: message.Username, UserID: message.UserID, Channel: message.Channel, Text: message.Text, ID: message.Post.Id, Extra: make(map[string][]interface{})}
+
+		// handle mattermost post properties (override username and attachments)
+		props := message.Post.Props
+		if props != nil {
+			if _, ok := props["override_username"].(string); ok {
+				rmsg.Username = props["override_username"].(string)
+			}
+			if _, ok := props["attachments"].([]interface{}); ok {
+				rmsg.Extra["attachments"] = props["attachments"].([]interface{})
+				if rmsg.Text == "" {
+					for _, attachment := range rmsg.Extra["attachments"] {
+						attach := attachment.(map[string]interface{})
+						if attach["text"].(string) != "" {
+							rmsg.Text += attach["text"].(string)
+							continue
+						}
+						if attach["fallback"].(string) != "" {
+							rmsg.Text += attach["fallback"].(string)
+						}
+					}
 				}
 			}
-			mchan <- m
 		}
+
+		// create a text for bridges that don't support native editing
+		if message.Raw.Event == "post_edited" && !b.GetBool("EditDisable") {
+			rmsg.Text = message.Text + b.GetString("EditSuffix")
+		}
+
+		if message.Raw.Event == "post_deleted" {
+			rmsg.Event = config.EVENT_MSG_DELETE
+		}
+
+		if len(message.Post.FileIds) > 0 {
+			for _, id := range message.Post.FileIds {
+				err := b.handleDownloadFile(rmsg, id)
+				if err != nil {
+					b.Log.Errorf("download failed: %s", err)
+				}
+			}
+		}
+		messages <- rmsg
 	}
 }
 
-func (b *Bmattermost) handleMatterHook(mchan chan *MMMessage) {
+func (b *Bmattermost) handleMatterHook(messages chan *config.Message) {
 	for {
 		message := b.mh.Receive()
-		flog.Debugf("Receiving from matterhook %#v", message)
-		m := &MMMessage{}
-		m.Username = message.UserName
-		m.Text = message.Text
-		m.Channel = message.ChannelName
-		mchan <- m
+		b.Log.Debugf("Receiving from matterhook %#v", message)
+		messages <- &config.Message{UserID: message.UserID, Username: message.UserName, Text: message.Text, Channel: message.ChannelName}
 	}
 }
+
+func (b *Bmattermost) apiLogin() error {
+	password := b.GetString("Password")
+	if b.GetString("Token") != "" {
+		password = "MMAUTHTOKEN=" + b.GetString("Token")
+	}
+
+	b.mc = matterclient.New(b.GetString("Login"), password, b.GetString("Team"), b.GetString("Server"))
+	if b.GetBool("debug") {
+		b.mc.SetLogLevel("debug")
+	}
+	b.mc.SkipTLSVerify = b.GetBool("SkipTLSVerify")
+	b.mc.NoTLS = b.GetBool("NoTLS")
+	b.Log.Infof("Connecting %s (team: %s) on %s", b.GetString("Login"), b.GetString("Team"), b.GetString("Server"))
+	err := b.mc.Login()
+	if err != nil {
+		return err
+	}
+	b.Log.Info("Connection succeeded")
+	b.TeamID = b.mc.GetTeamId()
+	go b.mc.WsReceiver()
+	go b.mc.StatusLoop()
+	return nil
+}
+
+// replaceAction replace the message with the correct action (/me) code
+func (b *Bmattermost) replaceAction(text string) (string, bool) {
+	if strings.HasPrefix(text, "*") && strings.HasSuffix(text, "*") {
+		return strings.Replace(text, "*", "", -1), true
+	}
+	return text, false
+}
+
+func (b *Bmattermost) cacheAvatar(msg *config.Message) (string, error) {
+	fi := msg.Extra["file"][0].(config.FileInfo)
+	/* if we have a sha we have successfully uploaded the file to the media server,
+	so we can now cache the sha */
+	if fi.SHA != "" {
+		b.Log.Debugf("Added %s to %s in avatarMap", fi.SHA, msg.UserID)
+		b.avatarMap[msg.UserID] = fi.SHA
+	}
+	return "", nil
+}
+
+// handleDownloadAvatar downloads the avatar of userid from channel
+// sends a EVENT_AVATAR_DOWNLOAD message to the gateway if successful.
+// logs an error message if it fails
+func (b *Bmattermost) handleDownloadAvatar(userid string, channel string) {
+	rmsg := config.Message{Username: "system", Text: "avatar", Channel: channel, Account: b.Account, UserID: userid, Event: config.EVENT_AVATAR_DOWNLOAD, Extra: make(map[string][]interface{})}
+	if _, ok := b.avatarMap[userid]; !ok {
+		data, resp := b.mc.Client.GetProfileImage(userid, "")
+		if resp.Error != nil {
+			b.Log.Errorf("ProfileImage download failed for %#v %s", userid, resp.Error)
+			return
+		}
+		err := helper.HandleDownloadSize(b.Log, &rmsg, userid+".png", int64(len(data)), b.General)
+		if err != nil {
+			b.Log.Error(err)
+			return
+		}
+		helper.HandleDownloadData(b.Log, &rmsg, userid+".png", rmsg.Text, "", &data, b.General)
+		b.Remote <- rmsg
+	}
+}
+
+// handleDownloadFile handles file download
+func (b *Bmattermost) handleDownloadFile(rmsg *config.Message, id string) error {
+	url, _ := b.mc.Client.GetFileLink(id)
+	finfo, resp := b.mc.Client.GetFileInfo(id)
+	if resp.Error != nil {
+		return resp.Error
+	}
+	err := helper.HandleDownloadSize(b.Log, rmsg, finfo.Name, finfo.Size, b.General)
+	if err != nil {
+		return err
+	}
+	data, resp := b.mc.Client.DownloadFile(id, true)
+	if resp.Error != nil {
+		return resp.Error
+	}
+	helper.HandleDownloadData(b.Log, rmsg, finfo.Name, rmsg.Text, url, &data, b.General)
+	return nil
+}
+
+// handleUploadFile handles native upload of files
+func (b *Bmattermost) handleUploadFile(msg *config.Message) (string, error) {
+	var err error
+	var res, id string
+	channelID := b.mc.GetChannelId(msg.Channel, "")
+	for _, f := range msg.Extra["file"] {
+		fi := f.(config.FileInfo)
+		id, err = b.mc.UploadFile(*fi.Data, channelID, fi.Name)
+		if err != nil {
+			return "", err
+		}
+		msg.Text = fi.Comment
+		if b.GetBool("PrefixMessagesWithNick") {
+			msg.Text = msg.Username + msg.Text
+		}
+		res, err = b.mc.PostMessageWithFiles(channelID, msg.Text, []string{id})
+	}
+	return res, err
+}
+
+// sendWebhook uses the configured WebhookURL to send the message
+func (b *Bmattermost) sendWebhook(msg config.Message) (string, error) {
+	// skip events
+	if msg.Event != "" {
+		return "", nil
+	}
+
+	if b.GetBool("PrefixMessagesWithNick") {
+		msg.Text = msg.Username + msg.Text
+	}
+	if msg.Extra != nil {
+		// this sends a message only if we received a config.EVENT_FILE_FAILURE_SIZE
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			iconURL := config.GetIconURL(&rmsg, b.GetString("iconurl"))
+			matterMessage := matterhook.OMessage{IconURL: iconURL, Channel: rmsg.Channel, UserName: rmsg.Username, Text: rmsg.Text, Props: make(map[string]interface{})}
+			matterMessage.Props["matterbridge_"+b.uuid] = true
+			b.mh.Send(matterMessage)
+		}
+
+		// webhook doesn't support file uploads, so we add the url manually
+		if len(msg.Extra["file"]) > 0 {
+			for _, f := range msg.Extra["file"] {
+				fi := f.(config.FileInfo)
+				if fi.URL != "" {
+					msg.Text += fi.URL
+				}
+			}
+		}
+	}
+
+	iconURL := config.GetIconURL(&msg, b.GetString("iconurl"))
+	matterMessage := matterhook.OMessage{IconURL: iconURL, Channel: msg.Channel, UserName: msg.Username, Text: msg.Text, Props: make(map[string]interface{})}
+	if msg.Avatar != "" {
+		matterMessage.IconURL = msg.Avatar
+	}
+	matterMessage.Props["matterbridge_"+b.uuid] = true
+	err := b.mh.Send(matterMessage)
+	if err != nil {
+		b.Log.Info(err)
+		return "", err
+	}
+	return "", nil
+}
+
+// skipMessages returns true if this message should not be handled
+func (b *Bmattermost) skipMessage(message *matterclient.Message) bool {
+	// Handle join/leave
+	if message.Type == "system_join_leave" ||
+		message.Type == "system_join_channel" ||
+		message.Type == "system_leave_channel" {
+		if b.GetBool("nosendjoinpart") {
+			return true
+		}
+		b.Log.Debugf("Sending JOIN_LEAVE event from %s to gateway", b.Account)
+		b.Remote <- config.Message{Username: "system", Text: message.Text, Channel: message.Channel, Account: b.Account, Event: config.EVENT_JOIN_LEAVE}
+		return true
+	}
+
+	// Handle edited messages
+	if (message.Raw.Event == "post_edited") && b.GetBool("EditDisable") {
+		return true
+	}
+
+	// Ignore messages sent from matterbridge
+	if message.Post.Props != nil {
+		if _, ok := message.Post.Props["matterbridge_"+b.uuid].(bool); ok {
+			b.Log.Debugf("sent by matterbridge, ignoring")
+			return true
+		}
+	}
+
+	// Ignore messages sent from a user logged in as the bot
+	if b.mc.User.Username == message.Username {
+		return true
+	}
+
+	// if the message has reactions don't repost it (for now, until we can correlate reaction with message)
+	if message.Post.HasReactions {
+		return true
+	}
+
+	// ignore messages from other teams than ours
+	if message.Raw.Data["team_id"].(string) != b.TeamID {
+		return true
+	}
+
+	// only handle posted, edited or deleted events
+	if !(message.Raw.Event == "posted" || message.Raw.Event == "post_edited" || message.Raw.Event == "post_deleted") {
+		return true
+	}
+	return false
+}

bridge/rocketchat/rocketchat.go

@@ -1,10 +1,11 @@
 package brocketchat
 
 import (
+	"github.com/42wim/matterbridge/bridge"
 	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
 	"github.com/42wim/matterbridge/hook/rockethook"
 	"github.com/42wim/matterbridge/matterhook"
-	log "github.com/Sirupsen/logrus"
 )
 
 type MMhook struct {
@@ -14,25 +15,11 @@ type MMhook struct {
 
 type Brocketchat struct {
 	MMhook
-	Config  *config.Protocol
-	Remote  chan config.Message
-	name    string
-	Account string
+	*bridge.Config
 }
 
-var flog *log.Entry
-var protocol = "rocketchat"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Brocketchat {
-	b := &Brocketchat{}
-	b.Config = &cfg
-	b.Remote = c
-	b.Account = account
-	return b
+func New(cfg *bridge.Config) bridge.Bridger {
+	return &Brocketchat{Config: cfg}
 }
 
 func (b *Brocketchat) Command(cmd string) string {
@@ -40,11 +27,11 @@ func (b *Brocketchat) Command(cmd string) string {
 }
 
 func (b *Brocketchat) Connect() error {
-	flog.Info("Connecting webhooks")
-	b.mh = matterhook.New(b.Config.URL,
-		matterhook.Config{InsecureSkipVerify: b.Config.SkipTLSVerify,
+	b.Log.Info("Connecting webhooks")
+	b.mh = matterhook.New(b.GetString("WebhookURL"),
+		matterhook.Config{InsecureSkipVerify: b.GetBool("SkipTLSVerify"),
 			DisableServer: true})
-	b.rh = rockethook.New(b.Config.URL, rockethook.Config{BindAddress: b.Config.BindAddress})
+	b.rh = rockethook.New(b.GetString("WebhookURL"), rockethook.Config{BindAddress: b.GetString("WebhookBindAddress")})
 	go b.handleRocketHook()
 	return nil
 }
@@ -54,34 +41,55 @@ func (b *Brocketchat) Disconnect() error {
 
 }
 
-func (b *Brocketchat) JoinChannel(channel string) error {
+func (b *Brocketchat) JoinChannel(channel config.ChannelInfo) error {
 	return nil
 }
 
-func (b *Brocketchat) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
-	matterMessage := matterhook.OMessage{IconURL: b.Config.IconURL}
+func (b *Brocketchat) Send(msg config.Message) (string, error) {
+	// ignore delete messages
+	if msg.Event == config.EVENT_MSG_DELETE {
+		return "", nil
+	}
+	b.Log.Debugf("=> Receiving %#v", msg)
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			iconURL := config.GetIconURL(&rmsg, b.GetString("iconurl"))
+			matterMessage := matterhook.OMessage{IconURL: iconURL, Channel: rmsg.Channel, UserName: rmsg.Username, Text: rmsg.Text}
+			b.mh.Send(matterMessage)
+		}
+		if len(msg.Extra["file"]) > 0 {
+			for _, f := range msg.Extra["file"] {
+				fi := f.(config.FileInfo)
+				if fi.URL != "" {
+					msg.Text += fi.URL
+				}
+			}
+		}
+	}
+
+	iconURL := config.GetIconURL(&msg, b.GetString("iconurl"))
+	matterMessage := matterhook.OMessage{IconURL: iconURL}
 	matterMessage.Channel = msg.Channel
 	matterMessage.UserName = msg.Username
 	matterMessage.Type = ""
 	matterMessage.Text = msg.Text
 	err := b.mh.Send(matterMessage)
 	if err != nil {
-		flog.Info(err)
-		return err
+		b.Log.Info(err)
+		return "", err
 	}
-	return nil
+	return "", nil
 }
 
 func (b *Brocketchat) handleRocketHook() {
 	for {
 		message := b.rh.Receive()
-		flog.Debugf("Receiving from rockethook %#v", message)
+		b.Log.Debugf("Receiving from rockethook %#v", message)
 		// do not loop
-		if message.UserName == b.Config.Nick {
+		if message.UserName == b.GetString("Nick") {
 			continue
 		}
-		flog.Debugf("Sending message from %s on %s to gateway", message.UserName, b.Account)
-		b.Remote <- config.Message{Text: message.Text, Username: message.UserName, Channel: message.ChannelName, Account: b.Account}
+		b.Log.Debugf("<= Sending message from %s on %s to gateway", message.UserName, b.Account)
+		b.Remote <- config.Message{Text: message.Text, Username: message.UserName, Channel: message.ChannelName, Account: b.Account, UserID: message.UserID}
 	}
 }

bridge/slack/slack.go

@@ -1,49 +1,40 @@
 package bslack
 
 import (
+	"bytes"
+	"errors"
 	"fmt"
-	"github.com/42wim/matterbridge/bridge/config"
-	"github.com/42wim/matterbridge/matterhook"
-	log "github.com/Sirupsen/logrus"
-	"github.com/nlopes/slack"
+	"html"
 	"regexp"
 	"strings"
+	"sync"
 	"time"
+
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
+	"github.com/42wim/matterbridge/matterhook"
+	"github.com/nlopes/slack"
+	"github.com/rs/xid"
 )
 
-type MMMessage struct {
-	Text     string
-	Channel  string
-	Username string
-	Raw      *slack.MessageEvent
-}
-
 type Bslack struct {
-	mh       *matterhook.Client
-	sc       *slack.Client
-	Config   *config.Protocol
-	rtm      *slack.RTM
-	Plus     bool
-	Remote   chan config.Message
-	Users    []slack.User
-	Account  string
-	si       *slack.Info
-	channels []slack.Channel
+	mh         *matterhook.Client
+	sc         *slack.Client
+	rtm        *slack.RTM
+	Users      []slack.User
+	Usergroups []slack.UserGroup
+	si         *slack.Info
+	channels   []slack.Channel
+	uuid       string
+	*bridge.Config
+	sync.RWMutex
 }
 
-var flog *log.Entry
-var protocol = "slack"
+const messageDeleted = "message_deleted"
 
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Bslack {
-	b := &Bslack{}
-	b.Config = &cfg
-	b.Remote = c
-	b.Account = account
-	return b
+func New(cfg *bridge.Config) bridge.Bridger {
+	return &Bslack{Config: cfg, uuid: xid.New().String()}
 }
 
 func (b *Bslack) Command(cmd string) string {
@@ -51,81 +42,174 @@ func (b *Bslack) Command(cmd string) string {
 }
 
 func (b *Bslack) Connect() error {
-	flog.Info("Connecting")
-	if !b.Config.UseAPI {
-		b.mh = matterhook.New(b.Config.URL,
-			matterhook.Config{BindAddress: b.Config.BindAddress})
-	} else {
-		b.sc = slack.New(b.Config.Token)
+	b.RLock()
+	defer b.RUnlock()
+	if b.GetString("WebhookBindAddress") != "" {
+		if b.GetString("WebhookURL") != "" {
+			b.Log.Info("Connecting using webhookurl (sending) and webhookbindaddress (receiving)")
+			b.mh = matterhook.New(b.GetString("WebhookURL"),
+				matterhook.Config{InsecureSkipVerify: b.GetBool("SkipTLSVerify"),
+					BindAddress: b.GetString("WebhookBindAddress")})
+		} else if b.GetString("Token") != "" {
+			b.Log.Info("Connecting using token (sending)")
+			b.sc = slack.New(b.GetString("Token"))
+			b.rtm = b.sc.NewRTM()
+			go b.rtm.ManageConnection()
+			b.Log.Info("Connecting using webhookbindaddress (receiving)")
+			b.mh = matterhook.New(b.GetString("WebhookURL"),
+				matterhook.Config{InsecureSkipVerify: b.GetBool("SkipTLSVerify"),
+					BindAddress: b.GetString("WebhookBindAddress")})
+		} else {
+			b.Log.Info("Connecting using webhookbindaddress (receiving)")
+			b.mh = matterhook.New(b.GetString("WebhookURL"),
+				matterhook.Config{InsecureSkipVerify: b.GetBool("SkipTLSVerify"),
+					BindAddress: b.GetString("WebhookBindAddress")})
+		}
+		go b.handleSlack()
+		return nil
+	}
+	if b.GetString("WebhookURL") != "" {
+		b.Log.Info("Connecting using webhookurl (sending)")
+		b.mh = matterhook.New(b.GetString("WebhookURL"),
+			matterhook.Config{InsecureSkipVerify: b.GetBool("SkipTLSVerify"),
+				DisableServer: true})
+		if b.GetString("Token") != "" {
+			b.Log.Info("Connecting using token (receiving)")
+			b.sc = slack.New(b.GetString("Token"))
+			b.rtm = b.sc.NewRTM()
+			go b.rtm.ManageConnection()
+			go b.handleSlack()
+		}
+	} else if b.GetString("Token") != "" {
+		b.Log.Info("Connecting using token (sending and receiving)")
+		b.sc = slack.New(b.GetString("Token"))
 		b.rtm = b.sc.NewRTM()
 		go b.rtm.ManageConnection()
+		go b.handleSlack()
+	}
+	if b.GetString("WebhookBindAddress") == "" && b.GetString("WebhookURL") == "" && b.GetString("Token") == "" {
+		return errors.New("no connection method found. See that you have WebhookBindAddress, WebhookURL or Token configured")
 	}
-	flog.Info("Connection succeeded")
-	go b.handleSlack()
 	return nil
 }
 
 func (b *Bslack) Disconnect() error {
-	return nil
-
+	return b.rtm.Disconnect()
 }
 
-func (b *Bslack) JoinChannel(channel string) error {
+func (b *Bslack) JoinChannel(channel config.ChannelInfo) error {
 	// we can only join channels using the API
-	if b.Config.UseAPI {
-		_, err := b.sc.JoinChannel(channel)
+	if b.sc != nil {
+		if strings.HasPrefix(b.GetString("Token"), "xoxb") {
+			// TODO check if bot has already joined channel
+			return nil
+		}
+		_, err := b.sc.JoinChannel(channel.Name)
 		if err != nil {
-			return err
+			switch err.Error() {
+			case "name_taken", "restricted_action":
+			case "default":
+				{
+					return err
+				}
+			}
 		}
 	}
 	return nil
 }
 
-func (b *Bslack) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
-	if msg.Account == b.Account {
-		return nil
+func (b *Bslack) Send(msg config.Message) (string, error) {
+	b.Log.Debugf("=> Receiving %#v", msg)
+
+	// Make a action /me of the message
+	if msg.Event == config.EVENT_USER_ACTION {
+		msg.Text = "_" + msg.Text + "_"
 	}
-	nick := msg.Username
-	message := msg.Text
-	channel := msg.Channel
-	if b.Config.PrefixMessagesWithNick {
-		message = nick + " " + message
+
+	// Use webhook to send the message
+	if b.GetString("WebhookURL") != "" {
+		return b.sendWebhook(msg)
 	}
-	if !b.Config.UseAPI {
-		matterMessage := matterhook.OMessage{IconURL: b.Config.IconURL}
-		matterMessage.Channel = channel
-		matterMessage.UserName = nick
-		matterMessage.Type = ""
-		matterMessage.Text = message
-		err := b.mh.Send(matterMessage)
-		if err != nil {
-			flog.Info(err)
-			return err
-		}
-		return nil
-	}
-	schannel, err := b.getChannelByName(channel)
+
+	// get the slack channel
+	schannel, err := b.getChannelByName(msg.Channel)
 	if err != nil {
-		return err
+		return "", err
 	}
+
+	// Delete message
+	if msg.Event == config.EVENT_MSG_DELETE {
+		// some protocols echo deletes, but with empty ID
+		if msg.ID == "" {
+			return "", nil
+		}
+		// we get a "slack <ID>", split it
+		ts := strings.Fields(msg.ID)
+		_, _, err := b.sc.DeleteMessage(schannel.ID, ts[1])
+		if err != nil {
+			return msg.ID, err
+		}
+		return msg.ID, nil
+	}
+
+	// Prepend nick if configured
+	if b.GetBool("PrefixMessagesWithNick") {
+		msg.Text = msg.Username + msg.Text
+	}
+
+	// Edit message if we have an ID
+	if msg.ID != "" {
+		ts := strings.Fields(msg.ID)
+		_, _, _, err := b.sc.UpdateMessage(schannel.ID, ts[1], msg.Text)
+		if err != nil {
+			return msg.ID, err
+		}
+		return msg.ID, nil
+	}
+
+	// create slack new post parameters
 	np := slack.NewPostMessageParameters()
-	if b.Config.PrefixMessagesWithNick == true {
+	if b.GetBool("PrefixMessagesWithNick") {
 		np.AsUser = true
 	}
-	np.Username = nick
-	np.IconURL = config.GetIconURL(&msg, b.Config)
+	np.Username = msg.Username
+	np.LinkNames = 1 // replace mentions
+	np.IconURL = config.GetIconURL(&msg, b.GetString("iconurl"))
 	if msg.Avatar != "" {
 		np.IconURL = msg.Avatar
 	}
-	b.sc.PostMessage(schannel.ID, message, np)
+	// add a callback ID so we can see we created it
+	np.Attachments = append(np.Attachments, slack.Attachment{CallbackID: "matterbridge_" + b.uuid})
+	// add file attachments
+	np.Attachments = append(np.Attachments, b.createAttach(msg.Extra)...)
+	// add slack attachments (from another slack bridge)
+	if len(msg.Extra["slack_attachment"]) > 0 {
+		for _, attach := range msg.Extra["slack_attachment"] {
+			np.Attachments = append(np.Attachments, attach.([]slack.Attachment)...)
+		}
+	}
 
-	/*
-	   newmsg := b.rtm.NewOutgoingMessage(message, schannel.ID)
-	   b.rtm.SendMessage(newmsg)
-	*/
+	// Upload a file if it exists
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.sc.PostMessage(schannel.ID, rmsg.Username+rmsg.Text, np)
+		}
+		// check if we have files to upload (from slack, telegram or mattermost)
+		if len(msg.Extra["file"]) > 0 {
+			b.handleUploadFile(&msg, schannel.ID)
+		}
+	}
 
-	return nil
+	// Post normal message
+	_, id, err := b.sc.PostMessage(schannel.ID, msg.Text, np)
+	if err != nil {
+		return "", err
+	}
+	return "slack " + id, nil
+}
+
+func (b *Bslack) Reload(cfg *bridge.Config) (string, error) {
+	return "", nil
 }
 
 func (b *Bslack) getAvatar(user string) string {
@@ -165,62 +249,61 @@ func (b *Bslack) getChannelByID(ID string) (*slack.Channel, error) {
 }
 
 func (b *Bslack) handleSlack() {
-	flog.Debugf("Choosing API based slack connection: %t", b.Config.UseAPI)
-	mchan := make(chan *MMMessage)
-	if b.Config.UseAPI {
-		go b.handleSlackClient(mchan)
+	messages := make(chan *config.Message)
+	if b.GetString("WebhookBindAddress") != "" {
+		b.Log.Debugf("Choosing webhooks based receiving")
+		go b.handleMatterHook(messages)
 	} else {
-		go b.handleMatterHook(mchan)
+		b.Log.Debugf("Choosing token based receiving")
+		go b.handleSlackClient(messages)
 	}
 	time.Sleep(time.Second)
-	flog.Debug("Start listening for Slack messages")
-	for message := range mchan {
-		// do not send messages from ourself
-		if b.Config.UseAPI && message.Username == b.si.User.Name {
-			continue
-		}
-		texts := strings.Split(message.Text, "\n")
-		for _, text := range texts {
-			flog.Debugf("Sending message from %s on %s to gateway", message.Username, b.Account)
-			b.Remote <- config.Message{Text: text, Username: message.Username, Channel: message.Channel, Account: b.Account, Avatar: b.getAvatar(message.Username)}
-		}
+	b.Log.Debug("Start listening for Slack messages")
+	for message := range messages {
+		b.Log.Debugf("<= Sending message from %s on %s to gateway", message.Username, b.Account)
+
+		// cleanup the message
+		message.Text = b.replaceMention(message.Text)
+		message.Text = b.replaceVariable(message.Text)
+		message.Text = b.replaceChannel(message.Text)
+		message.Text = b.replaceURL(message.Text)
+		message.Text = html.UnescapeString(message.Text)
+
+		// Add the avatar
+		message.Avatar = b.getAvatar(strings.ToLower(message.Username))
+
+		b.Log.Debugf("<= Message is %#v", message)
+		b.Remote <- *message
 	}
 }
 
-func (b *Bslack) handleSlackClient(mchan chan *MMMessage) {
-	count := 0
+func (b *Bslack) handleSlackClient(messages chan *config.Message) {
 	for msg := range b.rtm.IncomingEvents {
+		if msg.Type != "user_typing" && msg.Type != "latency_report" {
+			b.Log.Debugf("== Receiving event %#v", msg.Data)
+		}
 		switch ev := msg.Data.(type) {
 		case *slack.MessageEvent:
-			// ignore first message
-			if count > 0 {
-				flog.Debugf("Receiving from slackclient %#v", ev)
-				// use our own func because rtm.GetChannelInfo doesn't work for private channels
-				channel, err := b.getChannelByID(ev.Channel)
-				if err != nil {
-					continue
-				}
-				user, err := b.rtm.GetUserInfo(ev.User)
-				if err != nil {
-					continue
-				}
-				m := &MMMessage{}
-				m.Username = user.Name
-				m.Channel = channel.Name
-				m.Text = ev.Text
-				m.Raw = ev
-				m.Text = b.replaceMention(m.Text)
-				mchan <- m
+			if b.skipMessageEvent(ev) {
+				b.Log.Debugf("Skipped message: %#v", ev)
+				continue
 			}
-			count++
+			rmsg, err := b.handleMessageEvent(ev)
+			if err != nil {
+				b.Log.Errorf("%#v", err)
+				continue
+			}
+			messages <- rmsg
 		case *slack.OutgoingErrorEvent:
-			flog.Debugf("%#v", ev.Error())
+			b.Log.Debugf("%#v", ev.Error())
 		case *slack.ChannelJoinedEvent:
 			b.Users, _ = b.sc.GetUsers()
+			b.Usergroups, _ = b.sc.GetUserGroups()
 		case *slack.ConnectedEvent:
 			b.channels = ev.Info.Channels
 			b.si = ev.Info
 			b.Users, _ = b.sc.GetUsers()
+			b.Usergroups, _ = b.sc.GetUserGroups()
 			// add private channels
 			groups, _ := b.sc.GetGroups(true)
 			for _, g := range groups {
@@ -230,42 +313,372 @@ func (b *Bslack) handleSlackClient(mchan chan *MMMessage) {
 				b.channels = append(b.channels, *channel)
 			}
 		case *slack.InvalidAuthEvent:
-			flog.Fatalf("Invalid Token %#v", ev)
+			b.Log.Fatalf("Invalid Token %#v", ev)
+		case *slack.ConnectionErrorEvent:
+			b.Log.Errorf("Connection failed %#v %#v", ev.Error(), ev.ErrorObj)
 		default:
 		}
 	}
 }
 
-func (b *Bslack) handleMatterHook(mchan chan *MMMessage) {
+func (b *Bslack) handleMatterHook(messages chan *config.Message) {
 	for {
 		message := b.mh.Receive()
-		flog.Debugf("receiving from matterhook (slack) %#v", message)
-		m := &MMMessage{}
-		m.Username = message.UserName
-		m.Text = message.Text
-		m.Text = b.replaceMention(m.Text)
-		m.Channel = message.ChannelName
-		if m.Username == "slackbot" {
+		b.Log.Debugf("receiving from matterhook (slack) %#v", message)
+		if message.UserName == "slackbot" {
 			continue
 		}
-		mchan <- m
+		messages <- &config.Message{Username: message.UserName, Text: message.Text, Channel: message.ChannelName}
 	}
 }
 
 func (b *Bslack) userName(id string) string {
 	for _, u := range b.Users {
 		if u.ID == id {
+			if u.Profile.DisplayName != "" {
+				return u.Profile.DisplayName
+			}
 			return u.Name
 		}
 	}
 	return ""
 }
 
+/*
+func (b *Bslack) userGroupName(id string) string {
+	for _, u := range b.Usergroups {
+		if u.ID == id {
+			return u.Name
+		}
+	}
+	return ""
+}
+*/
+
+// @see https://api.slack.com/docs/message-formatting#linking_to_channels_and_users
 func (b *Bslack) replaceMention(text string) string {
-	results := regexp.MustCompile(`<@([a-zA-z0-9]+)>`).FindAllStringSubmatch(text, -1)
+	results := regexp.MustCompile(`<@([a-zA-Z0-9]+)>`).FindAllStringSubmatch(text, -1)
 	for _, r := range results {
 		text = strings.Replace(text, "<@"+r[1]+">", "@"+b.userName(r[1]), -1)
-
 	}
 	return text
 }
+
+// @see https://api.slack.com/docs/message-formatting#linking_to_channels_and_users
+func (b *Bslack) replaceChannel(text string) string {
+	results := regexp.MustCompile(`<#[a-zA-Z0-9]+\|(.+?)>`).FindAllStringSubmatch(text, -1)
+	for _, r := range results {
+		text = strings.Replace(text, r[0], "#"+r[1], -1)
+	}
+	return text
+}
+
+// @see https://api.slack.com/docs/message-formatting#variables
+func (b *Bslack) replaceVariable(text string) string {
+	results := regexp.MustCompile(`<!((?:subteam\^)?[a-zA-Z0-9]+)(?:\|@?(.+?))?>`).FindAllStringSubmatch(text, -1)
+	for _, r := range results {
+		if r[2] != "" {
+			text = strings.Replace(text, r[0], "@"+r[2], -1)
+		} else {
+			text = strings.Replace(text, r[0], "@"+r[1], -1)
+		}
+	}
+	return text
+}
+
+// @see https://api.slack.com/docs/message-formatting#linking_to_urls
+func (b *Bslack) replaceURL(text string) string {
+	results := regexp.MustCompile(`<(.*?)(\|.*?)?>`).FindAllStringSubmatch(text, -1)
+	for _, r := range results {
+		if len(strings.TrimSpace(r[2])) == 1 { // A display text separator was found, but the text was blank
+			text = strings.Replace(text, r[0], "", -1)
+		} else {
+			text = strings.Replace(text, r[0], r[1], -1)
+		}
+	}
+	return text
+}
+
+func (b *Bslack) createAttach(extra map[string][]interface{}) []slack.Attachment {
+	var attachs []slack.Attachment
+	for _, v := range extra["attachments"] {
+		entry := v.(map[string]interface{})
+		s := slack.Attachment{}
+		s.Fallback = entry["fallback"].(string)
+		s.Color = entry["color"].(string)
+		s.Pretext = entry["pretext"].(string)
+		s.AuthorName = entry["author_name"].(string)
+		s.AuthorLink = entry["author_link"].(string)
+		s.AuthorIcon = entry["author_icon"].(string)
+		s.Title = entry["title"].(string)
+		s.TitleLink = entry["title_link"].(string)
+		s.Text = entry["text"].(string)
+		s.ImageURL = entry["image_url"].(string)
+		s.ThumbURL = entry["thumb_url"].(string)
+		s.Footer = entry["footer"].(string)
+		s.FooterIcon = entry["footer_icon"].(string)
+		attachs = append(attachs, s)
+	}
+	return attachs
+}
+
+// handleDownloadFile handles file download
+func (b *Bslack) handleDownloadFile(rmsg *config.Message, file *slack.File) error {
+	// if we have a file attached, download it (in memory) and put a pointer to it in msg.Extra
+	// limit to 1MB for now
+	comment := ""
+	results := regexp.MustCompile(`.*?commented: (.*)`).FindAllStringSubmatch(rmsg.Text, -1)
+	if len(results) > 0 {
+		comment = results[0][1]
+	}
+
+	err := helper.HandleDownloadSize(b.Log, rmsg, file.Name, int64(file.Size), b.General)
+	if err != nil {
+		return err
+	}
+	// actually download the file
+	data, err := helper.DownloadFileAuth(file.URLPrivateDownload, "Bearer "+b.GetString("Token"))
+	if err != nil {
+		return fmt.Errorf("download %s failed %#v", file.URLPrivateDownload, err)
+	}
+	// add the downloaded data to the message
+	helper.HandleDownloadData(b.Log, rmsg, file.Name, comment, file.URLPrivateDownload, data, b.General)
+	return nil
+}
+
+// handleUploadFile handles native upload of files
+func (b *Bslack) handleUploadFile(msg *config.Message, channelID string) (string, error) {
+	var err error
+	for _, f := range msg.Extra["file"] {
+		fi := f.(config.FileInfo)
+		_, err = b.sc.UploadFile(slack.FileUploadParameters{
+			Reader:         bytes.NewReader(*fi.Data),
+			Filename:       fi.Name,
+			Channels:       []string{channelID},
+			InitialComment: fi.Comment,
+		})
+		if err != nil {
+			b.Log.Errorf("uploadfile %#v", err)
+		}
+	}
+	return "", nil
+}
+
+// handleMessageEvent handles the message events
+func (b *Bslack) handleMessageEvent(ev *slack.MessageEvent) (*config.Message, error) {
+	// update the userlist on a channel_join
+	if ev.SubType == "channel_join" {
+		b.Users, _ = b.sc.GetUsers()
+	}
+
+	// Edit message
+	if !b.GetBool("EditDisable") && ev.SubMessage != nil && ev.SubMessage.ThreadTimestamp != ev.SubMessage.Timestamp {
+		b.Log.Debugf("SubMessage %#v", ev.SubMessage)
+		ev.User = ev.SubMessage.User
+		ev.Text = ev.SubMessage.Text + b.GetString("EditSuffix")
+	}
+
+	// use our own func because rtm.GetChannelInfo doesn't work for private channels
+	channel, err := b.getChannelByID(ev.Channel)
+	if err != nil {
+		return nil, err
+	}
+
+	rmsg := config.Message{Text: ev.Text, Channel: channel.Name, Account: b.Account, ID: "slack " + ev.Timestamp, Extra: make(map[string][]interface{})}
+
+	// find the user id and name
+	if ev.User != "" && ev.SubType != messageDeleted && ev.SubType != "file_comment" {
+		user, err := b.rtm.GetUserInfo(ev.User)
+		if err != nil {
+			return nil, err
+		}
+		rmsg.UserID = user.ID
+		rmsg.Username = user.Name
+		if user.Profile.DisplayName != "" {
+			rmsg.Username = user.Profile.DisplayName
+		}
+	}
+
+	// See if we have some text in the attachments
+	if rmsg.Text == "" {
+		for _, attach := range ev.Attachments {
+			if attach.Text != "" {
+				if attach.Title != "" {
+					rmsg.Text = attach.Title + "\n"
+				}
+				rmsg.Text += attach.Text
+			} else {
+				rmsg.Text = attach.Fallback
+			}
+		}
+	}
+
+	// when using webhookURL we can't check if it's our webhook or not for now
+	if rmsg.Username == "" && ev.BotID != "" && b.GetString("WebhookURL") == "" {
+		bot, err := b.rtm.GetBotInfo(ev.BotID)
+		if err != nil {
+			return nil, err
+		}
+		if bot.Name != "" {
+			rmsg.Username = bot.Name
+			if ev.Username != "" {
+				rmsg.Username = ev.Username
+			}
+			rmsg.UserID = bot.ID
+		}
+
+		// fixes issues with matterircd users
+		if bot.Name == "Slack API Tester" {
+			user, err := b.rtm.GetUserInfo(ev.User)
+			if err != nil {
+				return nil, err
+			}
+			rmsg.UserID = user.ID
+			rmsg.Username = user.Name
+			if user.Profile.DisplayName != "" {
+				rmsg.Username = user.Profile.DisplayName
+			}
+		}
+	}
+
+	// file comments are set by the system (because there is no username given)
+	if ev.SubType == "file_comment" {
+		rmsg.Username = "system"
+	}
+
+	// do we have a /me action
+	if ev.SubType == "me_message" {
+		rmsg.Event = config.EVENT_USER_ACTION
+	}
+
+	// Handle join/leave
+	if ev.SubType == "channel_leave" || ev.SubType == "channel_join" {
+		rmsg.Username = "system"
+		rmsg.Event = config.EVENT_JOIN_LEAVE
+	}
+
+	// edited messages have a submessage, use this timestamp
+	if ev.SubMessage != nil {
+		rmsg.ID = "slack " + ev.SubMessage.Timestamp
+	}
+
+	// deleted message event
+	if ev.SubType == messageDeleted {
+		rmsg.Text = config.EVENT_MSG_DELETE
+		rmsg.Event = config.EVENT_MSG_DELETE
+		rmsg.ID = "slack " + ev.DeletedTimestamp
+	}
+
+	// topic change event
+	if ev.SubType == "channel_topic" || ev.SubType == "channel_purpose" {
+		rmsg.Event = config.EVENT_TOPIC_CHANGE
+	}
+
+	// Only deleted messages can have a empty username and text
+	if (rmsg.Text == "" || rmsg.Username == "") && ev.SubType != messageDeleted {
+		// this is probably a webhook we couldn't resolve
+		if ev.BotID != "" {
+			return nil, fmt.Errorf("probably an incoming webhook we couldn't resolve (maybe ourselves)")
+		}
+		return nil, fmt.Errorf("empty message and not a deleted message")
+	}
+
+	// save the attachments, so that we can send them to other slack (compatible) bridges
+	if len(ev.Attachments) > 0 {
+		rmsg.Extra["slack_attachment"] = append(rmsg.Extra["slack_attachment"], ev.Attachments)
+	}
+
+	// if we have a file attached, download it (in memory) and put a pointer to it in msg.Extra
+	if ev.File != nil {
+		err := b.handleDownloadFile(&rmsg, ev.File)
+		if err != nil {
+			b.Log.Errorf("download failed: %s", err)
+		}
+	}
+
+	return &rmsg, nil
+}
+
+// sendWebhook uses the configured WebhookURL to send the message
+func (b *Bslack) sendWebhook(msg config.Message) (string, error) {
+	// skip events
+	if msg.Event != "" {
+		return "", nil
+	}
+
+	if b.GetBool("PrefixMessagesWithNick") {
+		msg.Text = msg.Username + msg.Text
+	}
+
+	if msg.Extra != nil {
+		// this sends a message only if we received a config.EVENT_FILE_FAILURE_SIZE
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			iconURL := config.GetIconURL(&rmsg, b.GetString("iconurl"))
+			matterMessage := matterhook.OMessage{IconURL: iconURL, Channel: msg.Channel, UserName: rmsg.Username, Text: rmsg.Text}
+			b.mh.Send(matterMessage)
+		}
+
+		// webhook doesn't support file uploads, so we add the url manually
+		if len(msg.Extra["file"]) > 0 {
+			for _, f := range msg.Extra["file"] {
+				fi := f.(config.FileInfo)
+				if fi.URL != "" {
+					msg.Text += " " + fi.URL
+				}
+			}
+		}
+	}
+
+	// if we have native slack_attachments add them
+	var attachs []slack.Attachment
+	if len(msg.Extra["slack_attachment"]) > 0 {
+		for _, attach := range msg.Extra["slack_attachment"] {
+			attachs = append(attachs, attach.([]slack.Attachment)...)
+		}
+	}
+
+	iconURL := config.GetIconURL(&msg, b.GetString("iconurl"))
+	matterMessage := matterhook.OMessage{IconURL: iconURL, Attachments: attachs, Channel: msg.Channel, UserName: msg.Username, Text: msg.Text}
+	if msg.Avatar != "" {
+		matterMessage.IconURL = msg.Avatar
+	}
+	err := b.mh.Send(matterMessage)
+	if err != nil {
+		b.Log.Error(err)
+		return "", err
+	}
+	return "", nil
+}
+
+// skipMessageEvent skips event that need to be skipped :-)
+func (b *Bslack) skipMessageEvent(ev *slack.MessageEvent) bool {
+	if ev.SubType == "channel_leave" || ev.SubType == "channel_join" {
+		return b.GetBool("nosendjoinpart")
+	}
+
+	// ignore pinned items
+	if ev.SubType == "pinned_item" || ev.SubType == "unpinned_item" {
+		return true
+	}
+
+	// do not send messages from ourself
+	if b.GetString("WebhookURL") == "" && b.GetString("WebhookBindAddress") == "" && ev.Username == b.si.User.Name {
+		return true
+	}
+
+	// skip messages we made ourselves
+	if len(ev.Attachments) > 0 {
+		if ev.Attachments[0].CallbackID == "matterbridge_"+b.uuid {
+			return true
+		}
+	}
+
+	if !b.GetBool("EditDisable") && ev.SubMessage != nil && ev.SubMessage.ThreadTimestamp != ev.SubMessage.Timestamp {
+		// it seems ev.SubMessage.Edited == nil when slack unfurls
+		// do not forward these messages #266
+		if ev.SubMessage.Edited == nil {
+			return true
+		}
+	}
+	return false
+}

bridge/sshchat/sshchat.go

@@ -0,0 +1,141 @@
+package bsshchat
+
+import (
+	"bufio"
+	"io"
+	"strings"
+
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
+	"github.com/shazow/ssh-chat/sshd"
+	log "github.com/sirupsen/logrus"
+)
+
+type Bsshchat struct {
+	r *bufio.Scanner
+	w io.WriteCloser
+	*bridge.Config
+}
+
+func New(cfg *bridge.Config) bridge.Bridger {
+	return &Bsshchat{Config: cfg}
+}
+
+func (b *Bsshchat) Connect() error {
+	var err error
+	b.Log.Infof("Connecting %s", b.GetString("Server"))
+	go func() {
+		err = sshd.ConnectShell(b.GetString("Server"), b.GetString("Nick"), func(r io.Reader, w io.WriteCloser) error {
+			b.r = bufio.NewScanner(r)
+			b.w = w
+			b.r.Scan()
+			w.Write([]byte("/theme mono\r\n"))
+			b.handleSshChat()
+			return nil
+		})
+	}()
+	if err != nil {
+		b.Log.Debugf("%#v", err)
+		return err
+	}
+	b.Log.Info("Connection succeeded")
+	return nil
+}
+
+func (b *Bsshchat) Disconnect() error {
+	return nil
+}
+
+func (b *Bsshchat) JoinChannel(channel config.ChannelInfo) error {
+	return nil
+}
+
+func (b *Bsshchat) Send(msg config.Message) (string, error) {
+	// ignore delete messages
+	if msg.Event == config.EVENT_MSG_DELETE {
+		return "", nil
+	}
+	b.Log.Debugf("=> Receiving %#v", msg)
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.w.Write([]byte(rmsg.Username + rmsg.Text + "\r\n"))
+		}
+		if len(msg.Extra["file"]) > 0 {
+			for _, f := range msg.Extra["file"] {
+				fi := f.(config.FileInfo)
+				if fi.Comment != "" {
+					msg.Text += fi.Comment + ": "
+				}
+				if fi.URL != "" {
+					msg.Text = fi.URL
+					if fi.Comment != "" {
+						msg.Text = fi.Comment + ": " + fi.URL
+					}
+				}
+				b.w.Write([]byte(msg.Username + msg.Text))
+			}
+			return "", nil
+		}
+	}
+	b.w.Write([]byte(msg.Username + msg.Text + "\r\n"))
+	return "", nil
+}
+
+/*
+func (b *Bsshchat) sshchatKeepAlive() chan bool {
+	done := make(chan bool)
+	go func() {
+		ticker := time.NewTicker(90 * time.Second)
+		defer ticker.Stop()
+		for {
+			select {
+			case <-ticker.C:
+				b.Log.Debugf("PING")
+				err := b.xc.PingC2S("", "")
+				if err != nil {
+					b.Log.Debugf("PING failed %#v", err)
+				}
+			case <-done:
+				return
+			}
+		}
+	}()
+	return done
+}
+*/
+
+func stripPrompt(s string) string {
+	pos := strings.LastIndex(s, "\033[K")
+	if pos < 0 {
+		return s
+	}
+	return s[pos+3:]
+}
+
+func (b *Bsshchat) handleSshChat() error {
+	/*
+		done := b.sshchatKeepAlive()
+		defer close(done)
+	*/
+	wait := true
+	for {
+		if b.r.Scan() {
+			// ignore messages from ourselves
+			if !strings.Contains(b.r.Text(), "\033[K") {
+				continue
+			}
+			res := strings.Split(stripPrompt(b.r.Text()), ":")
+			if res[0] == "-> Set theme" {
+				wait = false
+				log.Debugf("mono found, allowing")
+				continue
+			}
+			if !wait {
+				b.Log.Debugf("<= Message %#v", res)
+				rmsg := config.Message{Username: res[0], Text: strings.Join(res[1:], ":"), Channel: "sshchat", Account: b.Account, UserID: "nick"}
+				b.Remote <- rmsg
+			}
+		}
+	}
+}

bridge/steam/steam.go

@@ -0,0 +1,183 @@
+package bsteam
+
+import (
+	"fmt"
+
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
+	"github.com/Philipp15b/go-steam"
+	"github.com/Philipp15b/go-steam/protocol/steamlang"
+	"github.com/Philipp15b/go-steam/steamid"
+	//"io/ioutil"
+	"strconv"
+	"sync"
+	"time"
+)
+
+type Bsteam struct {
+	c         *steam.Client
+	connected chan struct{}
+	userMap   map[steamid.SteamId]string
+	sync.RWMutex
+	*bridge.Config
+}
+
+func New(cfg *bridge.Config) bridge.Bridger {
+	b := &Bsteam{Config: cfg}
+	b.userMap = make(map[steamid.SteamId]string)
+	b.connected = make(chan struct{})
+	return b
+}
+
+func (b *Bsteam) Connect() error {
+	b.Log.Info("Connecting")
+	b.c = steam.NewClient()
+	go b.handleEvents()
+	go b.c.Connect()
+	select {
+	case <-b.connected:
+		b.Log.Info("Connection succeeded")
+	case <-time.After(time.Second * 30):
+		return fmt.Errorf("connection timed out")
+	}
+	return nil
+}
+
+func (b *Bsteam) Disconnect() error {
+	b.c.Disconnect()
+	return nil
+
+}
+
+func (b *Bsteam) JoinChannel(channel config.ChannelInfo) error {
+	id, err := steamid.NewId(channel.Name)
+	if err != nil {
+		return err
+	}
+	b.c.Social.JoinChat(id)
+	return nil
+}
+
+func (b *Bsteam) Send(msg config.Message) (string, error) {
+	// ignore delete messages
+	if msg.Event == config.EVENT_MSG_DELETE {
+		return "", nil
+	}
+	id, err := steamid.NewId(msg.Channel)
+	if err != nil {
+		return "", err
+	}
+
+	// Handle files
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.c.Social.SendMessage(id, steamlang.EChatEntryType_ChatMsg, rmsg.Username+rmsg.Text)
+		}
+		if len(msg.Extra["file"]) > 0 {
+			for _, f := range msg.Extra["file"] {
+				fi := f.(config.FileInfo)
+				if fi.Comment != "" {
+					msg.Text += fi.Comment + ": "
+				}
+				if fi.URL != "" {
+					msg.Text = fi.URL
+					if fi.Comment != "" {
+						msg.Text = fi.Comment + ": " + fi.URL
+					}
+				}
+				b.c.Social.SendMessage(id, steamlang.EChatEntryType_ChatMsg, msg.Username+msg.Text)
+			}
+			return "", nil
+		}
+	}
+
+	b.c.Social.SendMessage(id, steamlang.EChatEntryType_ChatMsg, msg.Username+msg.Text)
+	return "", nil
+}
+
+func (b *Bsteam) getNick(id steamid.SteamId) string {
+	b.RLock()
+	defer b.RUnlock()
+	if name, ok := b.userMap[id]; ok {
+		return name
+	}
+	return "unknown"
+}
+
+func (b *Bsteam) handleEvents() {
+	myLoginInfo := new(steam.LogOnDetails)
+	myLoginInfo.Username = b.GetString("Login")
+	myLoginInfo.Password = b.GetString("Password")
+	myLoginInfo.AuthCode = b.GetString("AuthCode")
+	// Attempt to read existing auth hash to avoid steam guard.
+	// Maybe works
+	//myLoginInfo.SentryFileHash, _ = ioutil.ReadFile("sentry")
+	for event := range b.c.Events() {
+		//b.Log.Info(event)
+		switch e := event.(type) {
+		case *steam.ChatMsgEvent:
+			b.Log.Debugf("Receiving ChatMsgEvent: %#v", e)
+			b.Log.Debugf("<= Sending message from %s on %s to gateway", b.getNick(e.ChatterId), b.Account)
+			var channel int64
+			if e.ChatRoomId == 0 {
+				channel = int64(e.ChatterId)
+			} else {
+				// for some reason we have to remove 0x18000000000000
+				channel = int64(e.ChatRoomId) - 0x18000000000000
+			}
+			msg := config.Message{Username: b.getNick(e.ChatterId), Text: e.Message, Channel: strconv.FormatInt(channel, 10), Account: b.Account, UserID: strconv.FormatInt(int64(e.ChatterId), 10)}
+			b.Remote <- msg
+		case *steam.PersonaStateEvent:
+			b.Log.Debugf("PersonaStateEvent: %#v\n", e)
+			b.Lock()
+			b.userMap[e.FriendId] = e.Name
+			b.Unlock()
+		case *steam.ConnectedEvent:
+			b.c.Auth.LogOn(myLoginInfo)
+		case *steam.MachineAuthUpdateEvent:
+			/*
+				b.Log.Info("authupdate", e)
+				b.Log.Info("hash", e.Hash)
+				ioutil.WriteFile("sentry", e.Hash, 0666)
+			*/
+		case *steam.LogOnFailedEvent:
+			b.Log.Info("Logon failed", e)
+			switch e.Result {
+			case steamlang.EResult_AccountLogonDeniedNeedTwoFactorCode:
+				{
+					b.Log.Info("Steam guard isn't letting me in! Enter 2FA code:")
+					var code string
+					fmt.Scanf("%s", &code)
+					myLoginInfo.TwoFactorCode = code
+				}
+			case steamlang.EResult_AccountLogonDenied:
+				{
+					b.Log.Info("Steam guard isn't letting me in! Enter auth code:")
+					var code string
+					fmt.Scanf("%s", &code)
+					myLoginInfo.AuthCode = code
+				}
+			default:
+				b.Log.Errorf("LogOnFailedEvent: %#v ", e.Result)
+				// TODO: Handle EResult_InvalidLoginAuthCode
+				return
+			}
+		case *steam.LoggedOnEvent:
+			b.Log.Debugf("LoggedOnEvent: %#v", e)
+			b.connected <- struct{}{}
+			b.Log.Debugf("setting online")
+			b.c.Social.SetPersonaState(steamlang.EPersonaState_Online)
+		case *steam.DisconnectedEvent:
+			b.Log.Info("Disconnected")
+			b.Log.Info("Attempting to reconnect...")
+			b.c.Connect()
+		case steam.FatalErrorEvent:
+			b.Log.Error(e)
+		case error:
+			b.Log.Error(e)
+		default:
+			b.Log.Debugf("unknown event %#v", e)
+		}
+	}
+}

bridge/telegram/html.go

@@ -2,15 +2,16 @@ package btelegram
 
 import (
 	"bytes"
-	"github.com/russross/blackfriday"
 	"html"
+
+	"github.com/russross/blackfriday"
 )
 
-type customHtml struct {
+type customHTML struct {
 	blackfriday.Renderer
 }
 
-func (options *customHtml) Paragraph(out *bytes.Buffer, text func() bool) {
+func (options *customHTML) Paragraph(out *bytes.Buffer, text func() bool) {
 	marker := out.Len()
 
 	if !text() {
@@ -20,32 +21,32 @@ func (options *customHtml) Paragraph(out *bytes.Buffer, text func() bool) {
 	out.WriteString("\n")
 }
 
-func (options *customHtml) BlockCode(out *bytes.Buffer, text []byte, lang string) {
+func (options *customHTML) BlockCode(out *bytes.Buffer, text []byte, lang string) {
 	out.WriteString("<pre>")
 
 	out.WriteString(html.EscapeString(string(text)))
 	out.WriteString("</pre>\n")
 }
 
-func (options *customHtml) Header(out *bytes.Buffer, text func() bool, level int, id string) {
+func (options *customHTML) Header(out *bytes.Buffer, text func() bool, level int, id string) {
 	options.Paragraph(out, text)
 }
 
-func (options *customHtml) HRule(out *bytes.Buffer) {
+func (options *customHTML) HRule(out *bytes.Buffer) {
 	out.WriteByte('\n')
 }
 
-func (options *customHtml) BlockQuote(out *bytes.Buffer, text []byte) {
+func (options *customHTML) BlockQuote(out *bytes.Buffer, text []byte) {
 	out.WriteString("> ")
 	out.Write(text)
 	out.WriteByte('\n')
 }
 
-func (options *customHtml) List(out *bytes.Buffer, text func() bool, flags int) {
+func (options *customHTML) List(out *bytes.Buffer, text func() bool, flags int) {
 	options.Paragraph(out, text)
 }
 
-func (options *customHtml) ListItem(out *bytes.Buffer, text []byte, flags int) {
+func (options *customHTML) ListItem(out *bytes.Buffer, text []byte, flags int) {
 	out.WriteString("- ")
 	out.Write(text)
 	out.WriteByte('\n')
@@ -53,7 +54,7 @@ func (options *customHtml) ListItem(out *bytes.Buffer, text []byte, flags int) {
 
 func makeHTML(input string) string {
 	return string(blackfriday.Markdown([]byte(input),
-		&customHtml{blackfriday.HtmlRenderer(blackfriday.HTML_USE_XHTML|blackfriday.HTML_SKIP_IMAGES, "", "")},
+		&customHTML{blackfriday.HtmlRenderer(blackfriday.HTML_USE_XHTML|blackfriday.HTML_SKIP_IMAGES, "", "")},
 		blackfriday.EXTENSION_NO_INTRA_EMPHASIS|
 			blackfriday.EXTENSION_FENCED_CODE|
 			blackfriday.EXTENSION_AUTOLINK|

bridge/telegram/telegram.go

@@ -1,113 +1,441 @@
 package btelegram
 
 import (
+	"html"
+	"regexp"
 	"strconv"
+	"strings"
 
+	"github.com/42wim/matterbridge/bridge"
 	"github.com/42wim/matterbridge/bridge/config"
-	log "github.com/Sirupsen/logrus"
+	"github.com/42wim/matterbridge/bridge/helper"
 	"github.com/go-telegram-bot-api/telegram-bot-api"
 )
 
 type Btelegram struct {
-	c       *tgbotapi.BotAPI
-	Config  *config.Protocol
-	Remote  chan config.Message
-	Account string
+	c *tgbotapi.BotAPI
+	*bridge.Config
+	avatarMap map[string]string // keep cache of userid and avatar sha
 }
 
-var flog *log.Entry
-var protocol = "telegram"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Btelegram {
-	b := &Btelegram{}
-	b.Config = &cfg
-	b.Remote = c
-	b.Account = account
-	return b
+func New(cfg *bridge.Config) bridge.Bridger {
+	return &Btelegram{Config: cfg, avatarMap: make(map[string]string)}
 }
 
 func (b *Btelegram) Connect() error {
 	var err error
-	flog.Info("Connecting")
-	b.c, err = tgbotapi.NewBotAPI(b.Config.Token)
+	b.Log.Info("Connecting")
+	b.c, err = tgbotapi.NewBotAPI(b.GetString("Token"))
 	if err != nil {
-		flog.Debugf("%#v", err)
+		b.Log.Debugf("%#v", err)
 		return err
 	}
-	updates, err := b.c.GetUpdatesChan(tgbotapi.NewUpdate(0))
+	u := tgbotapi.NewUpdate(0)
+	u.Timeout = 60
+	updates, err := b.c.GetUpdatesChan(u)
 	if err != nil {
-		flog.Debugf("%#v", err)
+		b.Log.Debugf("%#v", err)
 		return err
 	}
-	flog.Info("Connection succeeded")
+	b.Log.Info("Connection succeeded")
 	go b.handleRecv(updates)
 	return nil
 }
 
 func (b *Btelegram) Disconnect() error {
 	return nil
-
 }
 
-func (b *Btelegram) JoinChannel(channel string) error {
+func (b *Btelegram) JoinChannel(channel config.ChannelInfo) error {
 	return nil
 }
 
-func (b *Btelegram) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
+func (b *Btelegram) Send(msg config.Message) (string, error) {
+	b.Log.Debugf("=> Receiving %#v", msg)
+
+	// get the chatid
 	chatid, err := strconv.ParseInt(msg.Channel, 10, 64)
 	if err != nil {
-		return err
+		return "", err
 	}
 
-	if b.Config.MessageFormat == "HTML" {
+	// map the file SHA to our user (caches the avatar)
+	if msg.Event == config.EVENT_AVATAR_DOWNLOAD {
+		return b.cacheAvatar(&msg)
+	}
+
+	if b.GetString("MessageFormat") == "HTML" {
 		msg.Text = makeHTML(msg.Text)
 	}
-	m := tgbotapi.NewMessage(chatid, msg.Username+msg.Text)
-	if b.Config.MessageFormat == "HTML" {
-		m.ParseMode = tgbotapi.ModeHTML
+
+	// Delete message
+	if msg.Event == config.EVENT_MSG_DELETE {
+		if msg.ID == "" {
+			return "", nil
+		}
+		msgid, err := strconv.Atoi(msg.ID)
+		if err != nil {
+			return "", err
+		}
+		_, err = b.c.DeleteMessage(tgbotapi.DeleteMessageConfig{ChatID: chatid, MessageID: msgid})
+		return "", err
 	}
-	_, err = b.c.Send(m)
-	return err
+
+	// Upload a file if it exists
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.sendMessage(chatid, rmsg.Username, rmsg.Text)
+		}
+		// check if we have files to upload (from slack, telegram or mattermost)
+		if len(msg.Extra["file"]) > 0 {
+			b.handleUploadFile(&msg, chatid)
+		}
+	}
+
+	// edit the message if we have a msg ID
+	if msg.ID != "" {
+		msgid, err := strconv.Atoi(msg.ID)
+		if err != nil {
+			return "", err
+		}
+		if strings.ToLower(b.GetString("MessageFormat")) == "htmlnick" {
+			b.Log.Debug("Using mode HTML - nick only")
+			msg.Text = html.EscapeString(msg.Text)
+		}
+		m := tgbotapi.NewEditMessageText(chatid, msgid, msg.Username+msg.Text)
+		if b.GetString("MessageFormat") == "HTML" {
+			b.Log.Debug("Using mode HTML")
+			m.ParseMode = tgbotapi.ModeHTML
+		}
+		if b.GetString("MessageFormat") == "Markdown" {
+			b.Log.Debug("Using mode markdown")
+			m.ParseMode = tgbotapi.ModeMarkdown
+		}
+		if strings.ToLower(b.GetString("MessageFormat")) == "htmlnick" {
+			b.Log.Debug("Using mode HTML - nick only")
+			m.ParseMode = tgbotapi.ModeHTML
+		}
+		_, err = b.c.Send(m)
+		if err != nil {
+			return "", err
+		}
+		return "", nil
+	}
+
+	// Post normal message
+	return b.sendMessage(chatid, msg.Username, msg.Text)
 }
 
 func (b *Btelegram) handleRecv(updates <-chan tgbotapi.Update) {
-	username := ""
-	text := ""
-	channel := ""
 	for update := range updates {
+		b.Log.Debugf("== Receiving event: %#v", update.Message)
+
+		if update.Message == nil && update.ChannelPost == nil && update.EditedMessage == nil && update.EditedChannelPost == nil {
+			b.Log.Error("Getting nil messages, this shouldn't happen.")
+			continue
+		}
+
+		var message *tgbotapi.Message
+
+		rmsg := config.Message{Account: b.Account, Extra: make(map[string][]interface{})}
+
 		// handle channels
 		if update.ChannelPost != nil {
-			if update.ChannelPost.From != nil {
-				username = update.ChannelPost.From.FirstName
-				if username == "" {
-					username = update.ChannelPost.From.UserName
-				}
-			}
-			text = update.ChannelPost.Text
-			channel = strconv.FormatInt(update.ChannelPost.Chat.ID, 10)
+			message = update.ChannelPost
+			rmsg.Text = message.Text
 		}
+
+		// edited channel message
+		if update.EditedChannelPost != nil && !b.GetBool("EditDisable") {
+			message = update.EditedChannelPost
+			rmsg.Text = rmsg.Text + message.Text + b.GetString("EditSuffix")
+		}
+
 		// handle groups
 		if update.Message != nil {
-			if update.Message.From != nil {
-				username = update.Message.From.FirstName
-				if username == "" {
-					username = update.Message.From.UserName
+			message = update.Message
+			rmsg.Text = message.Text
+		}
+
+		// edited group message
+		if update.EditedMessage != nil && !b.GetBool("EditDisable") {
+			message = update.EditedMessage
+			rmsg.Text = rmsg.Text + message.Text + b.GetString("EditSuffix")
+		}
+
+		// set the ID's from the channel or group message
+		rmsg.ID = strconv.Itoa(message.MessageID)
+		rmsg.Channel = strconv.FormatInt(message.Chat.ID, 10)
+
+		// handle username
+		if message.From != nil {
+			rmsg.UserID = strconv.Itoa(message.From.ID)
+			if b.GetBool("UseFirstName") {
+				rmsg.Username = message.From.FirstName
+			}
+			if rmsg.Username == "" {
+				rmsg.Username = message.From.UserName
+				if rmsg.Username == "" {
+					rmsg.Username = message.From.FirstName
 				}
 			}
-			text = update.Message.Text
-			channel = strconv.FormatInt(update.Message.Chat.ID, 10)
+			// only download avatars if we have a place to upload them (configured mediaserver)
+			if b.General.MediaServerUpload != "" {
+				b.handleDownloadAvatar(message.From.ID, rmsg.Channel)
+			}
 		}
-		if username == "" {
-			username = "unknown"
+
+		// if we really didn't find a username, set it to unknown
+		if rmsg.Username == "" {
+			rmsg.Username = "unknown"
 		}
-		if text != "" {
-			flog.Debugf("Sending message from %s on %s to gateway", username, b.Account)
-			b.Remote <- config.Message{Username: username, Text: text, Channel: channel, Account: b.Account}
+
+		// handle any downloads
+		err := b.handleDownload(message, &rmsg)
+		if err != nil {
+			b.Log.Errorf("download failed: %s", err)
+		}
+
+		// handle forwarded messages
+		if message.ForwardFrom != nil {
+			usernameForward := ""
+			if b.GetBool("UseFirstName") {
+				usernameForward = message.ForwardFrom.FirstName
+			}
+			if usernameForward == "" {
+				usernameForward = message.ForwardFrom.UserName
+				if usernameForward == "" {
+					usernameForward = message.ForwardFrom.FirstName
+				}
+			}
+			if usernameForward == "" {
+				usernameForward = "unknown"
+			}
+			rmsg.Text = "Forwarded from " + usernameForward + ": " + rmsg.Text
+		}
+
+		// quote the previous message
+		if message.ReplyToMessage != nil {
+			usernameReply := ""
+			if message.ReplyToMessage.From != nil {
+				if b.GetBool("UseFirstName") {
+					usernameReply = message.ReplyToMessage.From.FirstName
+				}
+				if usernameReply == "" {
+					usernameReply = message.ReplyToMessage.From.UserName
+					if usernameReply == "" {
+						usernameReply = message.ReplyToMessage.From.FirstName
+					}
+				}
+			}
+			if usernameReply == "" {
+				usernameReply = "unknown"
+			}
+			if !b.GetBool("QuoteDisable") {
+				rmsg.Text = b.handleQuote(rmsg.Text, usernameReply, message.ReplyToMessage.Text)
+			}
+		}
+
+		if rmsg.Text != "" || len(rmsg.Extra) > 0 {
+			rmsg.Text = helper.RemoveEmptyNewLines(rmsg.Text)
+			// channels don't have (always?) user information. see #410
+			if message.From != nil {
+				rmsg.Avatar = helper.GetAvatar(b.avatarMap, strconv.Itoa(message.From.ID), b.General)
+			}
+
+			b.Log.Debugf("<= Sending message from %s on %s to gateway", rmsg.Username, b.Account)
+			b.Log.Debugf("<= Message is %#v", rmsg)
+			b.Remote <- rmsg
 		}
 	}
 }
+
+func (b *Btelegram) getFileDirectURL(id string) string {
+	res, err := b.c.GetFileDirectURL(id)
+	if err != nil {
+		return ""
+	}
+	return res
+}
+
+// handleDownloadAvatar downloads the avatar of userid from channel
+// sends a EVENT_AVATAR_DOWNLOAD message to the gateway if successful.
+// logs an error message if it fails
+func (b *Btelegram) handleDownloadAvatar(userid int, channel string) {
+	rmsg := config.Message{Username: "system", Text: "avatar", Channel: channel, Account: b.Account, UserID: strconv.Itoa(userid), Event: config.EVENT_AVATAR_DOWNLOAD, Extra: make(map[string][]interface{})}
+	if _, ok := b.avatarMap[strconv.Itoa(userid)]; !ok {
+		photos, err := b.c.GetUserProfilePhotos(tgbotapi.UserProfilePhotosConfig{UserID: userid, Limit: 1})
+		if err != nil {
+			b.Log.Errorf("Userprofile download failed for %#v %s", userid, err)
+		}
+
+		if len(photos.Photos) > 0 {
+			photo := photos.Photos[0][0]
+			url := b.getFileDirectURL(photo.FileID)
+			name := strconv.Itoa(userid) + ".png"
+			b.Log.Debugf("trying to download %#v fileid %#v with size %#v", name, photo.FileID, photo.FileSize)
+
+			err := helper.HandleDownloadSize(b.Log, &rmsg, name, int64(photo.FileSize), b.General)
+			if err != nil {
+				b.Log.Error(err)
+				return
+			}
+			data, err := helper.DownloadFile(url)
+			if err != nil {
+				b.Log.Errorf("download %s failed %#v", url, err)
+				return
+			}
+			helper.HandleDownloadData(b.Log, &rmsg, name, rmsg.Text, "", data, b.General)
+			b.Remote <- rmsg
+		}
+	}
+}
+
+// handleDownloadFile handles file download
+func (b *Btelegram) handleDownload(message *tgbotapi.Message, rmsg *config.Message) error {
+	size := 0
+	var url, name, text string
+
+	if message.Sticker != nil {
+		v := message.Sticker
+		size = v.FileSize
+		url = b.getFileDirectURL(v.FileID)
+		urlPart := strings.Split(url, "/")
+		name = urlPart[len(urlPart)-1]
+		if !strings.HasSuffix(name, ".webp") {
+			name = name + ".webp"
+		}
+		text = " " + url
+	}
+	if message.Video != nil {
+		v := message.Video
+		size = v.FileSize
+		url = b.getFileDirectURL(v.FileID)
+		urlPart := strings.Split(url, "/")
+		name = urlPart[len(urlPart)-1]
+		text = " " + url
+	}
+	if message.Photo != nil {
+		photos := *message.Photo
+		size = photos[len(photos)-1].FileSize
+		url = b.getFileDirectURL(photos[len(photos)-1].FileID)
+		urlPart := strings.Split(url, "/")
+		name = urlPart[len(urlPart)-1]
+		text = " " + url
+	}
+	if message.Document != nil {
+		v := message.Document
+		size = v.FileSize
+		url = b.getFileDirectURL(v.FileID)
+		name = v.FileName
+		text = " " + v.FileName + " : " + url
+	}
+	if message.Voice != nil {
+		v := message.Voice
+		size = v.FileSize
+		url = b.getFileDirectURL(v.FileID)
+		urlPart := strings.Split(url, "/")
+		name = urlPart[len(urlPart)-1]
+		text = " " + url
+		if !strings.HasSuffix(name, ".ogg") {
+			name = name + ".ogg"
+		}
+	}
+	if message.Audio != nil {
+		v := message.Audio
+		size = v.FileSize
+		url = b.getFileDirectURL(v.FileID)
+		urlPart := strings.Split(url, "/")
+		name = urlPart[len(urlPart)-1]
+		text = " " + url
+	}
+	// if name is empty we didn't match a thing to download
+	if name == "" {
+		return nil
+	}
+	// use the URL instead of native upload
+	if b.GetBool("UseInsecureURL") {
+		b.Log.Debugf("Setting message text to :%s", text)
+		rmsg.Text = rmsg.Text + text
+		return nil
+	}
+	// if we have a file attached, download it (in memory) and put a pointer to it in msg.Extra
+	err := helper.HandleDownloadSize(b.Log, rmsg, name, int64(size), b.General)
+	if err != nil {
+		return err
+	}
+	data, err := helper.DownloadFile(url)
+	if err != nil {
+		return err
+	}
+	helper.HandleDownloadData(b.Log, rmsg, name, message.Caption, "", data, b.General)
+	return nil
+}
+
+// handleUploadFile handles native upload of files
+func (b *Btelegram) handleUploadFile(msg *config.Message, chatid int64) (string, error) {
+	var c tgbotapi.Chattable
+	for _, f := range msg.Extra["file"] {
+		fi := f.(config.FileInfo)
+		file := tgbotapi.FileBytes{fi.Name, *fi.Data}
+		re := regexp.MustCompile(".(jpg|png)$")
+		if re.MatchString(fi.Name) {
+			c = tgbotapi.NewPhotoUpload(chatid, file)
+		} else {
+			c = tgbotapi.NewDocumentUpload(chatid, file)
+		}
+		_, err := b.c.Send(c)
+		if err != nil {
+			b.Log.Errorf("file upload failed: %#v", err)
+		}
+		if fi.Comment != "" {
+			b.sendMessage(chatid, msg.Username, fi.Comment)
+		}
+	}
+	return "", nil
+}
+
+func (b *Btelegram) sendMessage(chatid int64, username, text string) (string, error) {
+	m := tgbotapi.NewMessage(chatid, "")
+	m.Text = username + text
+	if b.GetString("MessageFormat") == "HTML" {
+		b.Log.Debug("Using mode HTML")
+		m.ParseMode = tgbotapi.ModeHTML
+	}
+	if b.GetString("MessageFormat") == "Markdown" {
+		b.Log.Debug("Using mode markdown")
+		m.ParseMode = tgbotapi.ModeMarkdown
+	}
+	if strings.ToLower(b.GetString("MessageFormat")) == "htmlnick" {
+		b.Log.Debug("Using mode HTML - nick only")
+		m.Text = username + html.EscapeString(text)
+		m.ParseMode = tgbotapi.ModeHTML
+	}
+	res, err := b.c.Send(m)
+	if err != nil {
+		return "", err
+	}
+	return strconv.Itoa(res.MessageID), nil
+}
+
+func (b *Btelegram) cacheAvatar(msg *config.Message) (string, error) {
+	fi := msg.Extra["file"][0].(config.FileInfo)
+	/* if we have a sha we have successfully uploaded the file to the media server,
+	so we can now cache the sha */
+	if fi.SHA != "" {
+		b.Log.Debugf("Added %s to %s in avatarMap", fi.SHA, msg.UserID)
+		b.avatarMap[msg.UserID] = fi.SHA
+	}
+	return "", nil
+}
+
+func (b *Btelegram) handleQuote(message, quoteNick, quoteMessage string) string {
+	format := b.GetString("quoteformat")
+	if format == "" {
+		format = "{MESSAGE} (re @{QUOTENICK}: {QUOTEMESSAGE})"
+	}
+	format = strings.Replace(format, "{MESSAGE}", message, -1)
+	format = strings.Replace(format, "{QUOTENICK}", quoteNick, -1)
+	format = strings.Replace(format, "{QUOTEMESSAGE}", quoteMessage, -1)
+	return format
+}

bridge/xmpp/xmpp.go

@@ -2,48 +2,61 @@ package bxmpp
 
 import (
 	"crypto/tls"
-	"github.com/42wim/matterbridge/bridge/config"
-	log "github.com/Sirupsen/logrus"
-	"github.com/mattn/go-xmpp"
-
 	"strings"
 	"time"
+
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
+	"github.com/jpillora/backoff"
+	"github.com/matterbridge/go-xmpp"
+	"github.com/rs/xid"
 )
 
 type Bxmpp struct {
 	xc      *xmpp.Client
 	xmppMap map[string]string
-	Config  *config.Protocol
-	Remote  chan config.Message
-	Account string
+	*bridge.Config
 }
 
-var flog *log.Entry
-var protocol = "xmpp"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Bxmpp {
-	b := &Bxmpp{}
+func New(cfg *bridge.Config) bridge.Bridger {
+	b := &Bxmpp{Config: cfg}
 	b.xmppMap = make(map[string]string)
-	b.Config = &cfg
-	b.Account = account
-	b.Remote = c
 	return b
 }
 
 func (b *Bxmpp) Connect() error {
 	var err error
-	flog.Infof("Connecting %s", b.Config.Server)
+	b.Log.Infof("Connecting %s", b.GetString("Server"))
 	b.xc, err = b.createXMPP()
 	if err != nil {
-		flog.Debugf("%#v", err)
+		b.Log.Debugf("%#v", err)
 		return err
 	}
-	flog.Info("Connection succeeded")
-	go b.handleXmpp()
+	b.Log.Info("Connection succeeded")
+	go func() {
+		initial := true
+		bf := &backoff.Backoff{
+			Min:    time.Second,
+			Max:    5 * time.Minute,
+			Jitter: true,
+		}
+		for {
+			if initial {
+				b.handleXMPP()
+				initial = false
+			}
+			d := bf.Duration()
+			b.Log.Infof("Disconnected. Reconnecting in %s", d)
+			time.Sleep(d)
+			b.xc, err = b.createXMPP()
+			if err == nil {
+				b.Remote <- config.Message{Username: "system", Text: "rejoin", Channel: "", Account: b.Account, Event: config.EVENT_REJOIN_CHANNELS}
+				b.handleXMPP()
+				bf.Reset()
+			}
+		}
+	}()
 	return nil
 }
 
@@ -51,37 +64,66 @@ func (b *Bxmpp) Disconnect() error {
 	return nil
 }
 
-func (b *Bxmpp) JoinChannel(channel string) error {
-	b.xc.JoinMUCNoHistory(channel+"@"+b.Config.Muc, b.Config.Nick)
+func (b *Bxmpp) JoinChannel(channel config.ChannelInfo) error {
+	if channel.Options.Key != "" {
+		b.Log.Debugf("using key %s for channel %s", channel.Options.Key, channel.Name)
+		b.xc.JoinProtectedMUC(channel.Name+"@"+b.GetString("Muc"), b.GetString("Nick"), channel.Options.Key, xmpp.NoHistory, 0, nil)
+	} else {
+		b.xc.JoinMUCNoHistory(channel.Name+"@"+b.GetString("Muc"), b.GetString("Nick"))
+	}
 	return nil
 }
 
-func (b *Bxmpp) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
-	b.xc.Send(xmpp.Chat{Type: "groupchat", Remote: msg.Channel + "@" + b.Config.Muc, Text: msg.Username + msg.Text})
-	return nil
+func (b *Bxmpp) Send(msg config.Message) (string, error) {
+	var msgid = ""
+	var msgreplaceid = ""
+	// ignore delete messages
+	if msg.Event == config.EVENT_MSG_DELETE {
+		return "", nil
+	}
+	b.Log.Debugf("=> Receiving %#v", msg)
+
+	// Upload a file (in xmpp case send the upload URL because xmpp has no native upload support)
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.xc.Send(xmpp.Chat{Type: "groupchat", Remote: rmsg.Channel + "@" + b.GetString("Muc"), Text: rmsg.Username + rmsg.Text})
+		}
+		if len(msg.Extra["file"]) > 0 {
+			return b.handleUploadFile(&msg)
+		}
+	}
+
+	msgid = xid.New().String()
+	if msg.ID != "" {
+		msgid = msg.ID
+		msgreplaceid = msg.ID
+	}
+	// Post normal message
+	_, err := b.xc.Send(xmpp.Chat{Type: "groupchat", Remote: msg.Channel + "@" + b.GetString("Muc"), Text: msg.Username + msg.Text, ID: msgid, ReplaceID: msgreplaceid})
+	if err != nil {
+		return "", err
+	}
+	return msgid, nil
 }
 
 func (b *Bxmpp) createXMPP() (*xmpp.Client, error) {
 	tc := new(tls.Config)
-	tc.InsecureSkipVerify = b.Config.SkipTLSVerify
-	tc.ServerName = strings.Split(b.Config.Server, ":")[0]
+	tc.InsecureSkipVerify = b.GetBool("SkipTLSVerify")
+	tc.ServerName = strings.Split(b.GetString("Server"), ":")[0]
 	options := xmpp.Options{
-		Host:      b.Config.Server,
-		User:      b.Config.Jid,
-		Password:  b.Config.Password,
-		NoTLS:     true,
-		StartTLS:  true,
-		TLSConfig: tc,
-
-		//StartTLS:      false,
-		Debug:                        true,
+		Host:                         b.GetString("Server"),
+		User:                         b.GetString("Jid"),
+		Password:                     b.GetString("Password"),
+		NoTLS:                        true,
+		StartTLS:                     true,
+		TLSConfig:                    tc,
+		Debug:                        b.GetBool("debug"),
+		Logger:                       b.Log.Writer(),
 		Session:                      true,
 		Status:                       "",
 		StatusMessage:                "",
 		Resource:                     "",
 		InsecureAllowUnencryptedAuth: false,
-		//InsecureAllowUnencryptedAuth: true,
 	}
 	var err error
 	b.xc, err = options.NewClient()
@@ -96,7 +138,11 @@ func (b *Bxmpp) xmppKeepAlive() chan bool {
 		for {
 			select {
 			case <-ticker.C:
-				b.xc.PingC2S("", "")
+				b.Log.Debugf("PING")
+				err := b.xc.PingC2S("", "")
+				if err != nil {
+					b.Log.Debugf("PING failed %#v", err)
+				}
 			case <-done:
 				return
 			}
@@ -105,10 +151,11 @@ func (b *Bxmpp) xmppKeepAlive() chan bool {
 	return done
 }
 
-func (b *Bxmpp) handleXmpp() error {
+func (b *Bxmpp) handleXMPP() error {
+	var ok bool
+	var msgid string
 	done := b.xmppKeepAlive()
 	defer close(done)
-	nodelay := time.Time{}
 	for {
 		m, err := b.xc.Recv()
 		if err != nil {
@@ -116,23 +163,104 @@ func (b *Bxmpp) handleXmpp() error {
 		}
 		switch v := m.(type) {
 		case xmpp.Chat:
-			var channel, nick string
 			if v.Type == "groupchat" {
-				s := strings.Split(v.Remote, "@")
-				if len(s) == 2 {
-					channel = s[0]
+				b.Log.Debugf("== Receiving %#v", v)
+				// skip invalid messages
+				if b.skipMessage(v) {
+					continue
 				}
-				s = strings.Split(s[1], "/")
-				if len(s) == 2 {
-					nick = s[1]
+				msgid = v.ID
+				if v.ReplaceID != "" {
+					msgid = v.ReplaceID
 				}
-				if nick != b.Config.Nick && v.Stamp == nodelay && v.Text != "" {
-					flog.Debugf("Sending message from %s on %s to gateway", nick, b.Account)
-					b.Remote <- config.Message{Username: nick, Text: v.Text, Channel: channel, Account: b.Account}
+				rmsg := config.Message{Username: b.parseNick(v.Remote), Text: v.Text, Channel: b.parseChannel(v.Remote), Account: b.Account, UserID: v.Remote, ID: msgid}
+
+				// check if we have an action event
+				rmsg.Text, ok = b.replaceAction(rmsg.Text)
+				if ok {
+					rmsg.Event = config.EVENT_USER_ACTION
 				}
+				b.Log.Debugf("<= Sending message from %s on %s to gateway", rmsg.Username, b.Account)
+				b.Log.Debugf("<= Message is %#v", rmsg)
+				b.Remote <- rmsg
 			}
 		case xmpp.Presence:
 			// do nothing
 		}
 	}
 }
+
+func (b *Bxmpp) replaceAction(text string) (string, bool) {
+	if strings.HasPrefix(text, "/me ") {
+		return strings.Replace(text, "/me ", "", -1), true
+	}
+	return text, false
+}
+
+// handleUploadFile handles native upload of files
+func (b *Bxmpp) handleUploadFile(msg *config.Message) (string, error) {
+	var urldesc = ""
+
+	for _, f := range msg.Extra["file"] {
+		fi := f.(config.FileInfo)
+		if fi.Comment != "" {
+			msg.Text += fi.Comment + ": "
+		}
+		if fi.URL != "" {
+			msg.Text = fi.URL
+			if fi.Comment != "" {
+				msg.Text = fi.Comment + ": " + fi.URL
+				urldesc = fi.Comment
+			}
+		}
+		_, err := b.xc.Send(xmpp.Chat{Type: "groupchat", Remote: msg.Channel + "@" + b.GetString("Muc"), Text: msg.Username + msg.Text})
+		if err != nil {
+			return "", err
+		}
+		if fi.URL != "" {
+			b.xc.SendOOB(xmpp.Chat{Type: "groupchat", Remote: msg.Channel + "@" + b.GetString("Muc"), Ooburl: fi.URL, Oobdesc: urldesc})
+		}
+	}
+	return "", nil
+}
+
+func (b *Bxmpp) parseNick(remote string) string {
+	s := strings.Split(remote, "@")
+	if len(s) > 0 {
+		s = strings.Split(s[1], "/")
+		if len(s) == 2 {
+			return s[1] // nick
+		}
+	}
+	return ""
+}
+
+func (b *Bxmpp) parseChannel(remote string) string {
+	s := strings.Split(remote, "@")
+	if len(s) >= 2 {
+		return s[0] // channel
+	}
+	return ""
+}
+
+// skipMessage skips messages that need to be skipped
+func (b *Bxmpp) skipMessage(message xmpp.Chat) bool {
+	// skip messages from ourselves
+	if b.parseNick(message.Remote) == b.GetString("Nick") {
+		return true
+	}
+
+	// skip empty messages
+	if message.Text == "" {
+		return true
+	}
+
+	// skip subject messages
+	if strings.Contains(message.Text, "</subject>") {
+		return true
+	}
+
+	// skip delayed messages
+	t := time.Time{}
+	return message.Stamp != t
+}

bridge/zulip/zulip.go

@@ -0,0 +1,170 @@
+package bzulip
+
+import (
+	"encoding/json"
+	"io/ioutil"
+	"strconv"
+	"time"
+
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
+	gzb "github.com/matterbridge/gozulipbot"
+)
+
+type Bzulip struct {
+	q       *gzb.Queue
+	bot     *gzb.Bot
+	streams map[int]string
+	*bridge.Config
+}
+
+func New(cfg *bridge.Config) bridge.Bridger {
+	return &Bzulip{Config: cfg, streams: make(map[int]string)}
+}
+
+func (b *Bzulip) Connect() error {
+	bot := gzb.Bot{APIKey: b.GetString("token"), APIURL: b.GetString("server") + "/api/v1/", Email: b.GetString("login")}
+	bot.Init()
+	q, err := bot.RegisterAll()
+	b.q = q
+	b.bot = &bot
+	if err != nil {
+		b.Log.Errorf("Connect() %#v", err)
+		return err
+	}
+	// init stream
+	b.getChannel(0)
+	b.Log.Info("Connection succeeded")
+	go b.handleQueue()
+	return nil
+}
+
+func (b *Bzulip) Disconnect() error {
+	return nil
+}
+
+func (b *Bzulip) JoinChannel(channel config.ChannelInfo) error {
+	return nil
+}
+
+func (b *Bzulip) Send(msg config.Message) (string, error) {
+	b.Log.Debugf("=> Receiving %#v", msg)
+
+	// Delete message
+	if msg.Event == config.EVENT_MSG_DELETE {
+		if msg.ID == "" {
+			return "", nil
+		}
+		_, err := b.bot.UpdateMessage(msg.ID, "")
+		return "", err
+	}
+
+	// Upload a file if it exists
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.sendMessage(rmsg)
+		}
+		if len(msg.Extra["file"]) > 0 {
+			return b.handleUploadFile(&msg)
+		}
+	}
+
+	// edit the message if we have a msg ID
+	if msg.ID != "" {
+		_, err := b.bot.UpdateMessage(msg.ID, msg.Username+msg.Text)
+		return "", err
+	}
+
+	// Post normal message
+	return b.sendMessage(msg)
+}
+
+func (b *Bzulip) getChannel(id int) string {
+	if name, ok := b.streams[id]; ok {
+		return name
+	}
+	streams, err := b.bot.GetRawStreams()
+	if err != nil {
+		b.Log.Errorf("getChannel: %#v", err)
+		return ""
+	}
+	for _, stream := range streams.Streams {
+		b.streams[stream.StreamID] = stream.Name
+	}
+	if name, ok := b.streams[id]; ok {
+		return name
+	}
+	return ""
+}
+
+func (b *Bzulip) handleQueue() error {
+	for {
+		messages, _ := b.q.GetEvents()
+		for _, m := range messages {
+			b.Log.Debugf("== Receiving %#v", m)
+			// ignore our own messages
+			if m.SenderEmail == b.GetString("login") {
+				continue
+			}
+			rmsg := config.Message{Username: m.SenderFullName, Text: m.Content, Channel: b.getChannel(m.StreamID), Account: b.Account, UserID: strconv.Itoa(m.SenderID), Avatar: m.AvatarURL}
+			b.Log.Debugf("<= Sending message from %s on %s to gateway", rmsg.Username, b.Account)
+			b.Log.Debugf("<= Message is %#v", rmsg)
+			b.Remote <- rmsg
+			b.q.LastEventID = m.ID
+		}
+		time.Sleep(time.Second * 3)
+	}
+}
+
+func (b *Bzulip) sendMessage(msg config.Message) (string, error) {
+	topic := "matterbridge"
+	if b.GetString("topic") != "" {
+		topic = b.GetString("topic")
+	}
+	m := gzb.Message{
+		Stream:  msg.Channel,
+		Topic:   topic,
+		Content: msg.Username + msg.Text,
+	}
+	resp, err := b.bot.Message(m)
+	if err != nil {
+		return "", err
+	}
+	if resp != nil {
+		defer resp.Body.Close()
+		res, err := ioutil.ReadAll(resp.Body)
+		if err != nil {
+			return "", err
+		}
+		var jr struct {
+			ID int `json:"id"`
+		}
+		err = json.Unmarshal(res, &jr)
+		if err != nil {
+			return "", err
+		}
+		return strconv.Itoa(jr.ID), nil
+	}
+	return "", nil
+}
+
+func (b *Bzulip) handleUploadFile(msg *config.Message) (string, error) {
+	for _, f := range msg.Extra["file"] {
+		fi := f.(config.FileInfo)
+		if fi.Comment != "" {
+			msg.Text += fi.Comment + ": "
+		}
+		if fi.URL != "" {
+			msg.Text = fi.URL
+			if fi.Comment != "" {
+				msg.Text = fi.Comment + ": " + fi.URL
+			}
+		}
+		_, err := b.sendMessage(*msg)
+		if err != nil {
+			return "", err
+		}
+	}
+	return "", nil
+}

changelog.md

@@ -1,3 +1,489 @@
+# v1.11.0
+
+## New features
+* general: Add config option MediaDownloadPath (#443). See `MediaDownloadPath` in matterbridge.toml.sample
+* general: Add MediaDownloadBlacklist option. Closes #442. See `MediaDownloadBlacklist` in matterbridge.toml.sample
+* xmpp: Add channel password support for XMPP (#451)
+* xmpp: Add message correction support for XMPP (#437)
+* telegram: Add support for MessageFormat=htmlnick (telegram). #444
+* mattermost: Add support for mattermost 5.x
+
+## Enhancements
+* slack: Add Title from attachment slack message (#446)
+* irc: Prevent white or black color codes (irc) (#434)
+
+## Bugfix
+* slack: Fix regexp in replaceMention (slack). (#435)
+* irc: Reconnect on quit. (irc) See #431 (#445)
+* sshchat: Ignore messages from ourself. (sshchat) Closes #439
+
+# v1.10.1
+## New features
+* irc: Colorize username sent to IRC using its crc32 IEEE checksum (#423). See `ColorNicks` in matterbridge.toml.sample
+* irc: Add support for CJK to/from utf-8 (irc). #400
+* telegram: Add QuoteFormat option (telegram). Closes #413. See `QuoteFormat` in matterbridge.toml.sample
+* xmpp: Send attached files to XMPP in different message with OOB data and without body (#421)
+
+## Bugfix
+* general: updated irc/xmpp/telegram libraries
+* mattermost/slack/rocketchat: Fix iconurl regression. Closes #430
+* mattermost/slack: Use uuid instead of userid. Fixes #429
+* slack: Avatar spoofing from Slack to Discord with uppercase in nick doesn't work (#433)
+* irc: Fix format string bug (irc) (#428)
+
+# v1.10.0
+## New features
+* general: Add support for reloading all settings automatically after changing config except connection and gateway configuration. Closes #373
+* zulip: New protocol support added (https://zulipchat.com)
+
+## Enhancements
+* general: Handle file comment better
+* steam: Handle file uploads to mediaserver (steam)
+* slack: Properly set Slack user who initiated slash command (#394)
+
+## Bugfix
+* general: Use only alphanumeric for file uploads to mediaserver. Closes #416
+* general: Fix crash on invalid filenames
+* general: Fix regression in ReplaceMessages and ReplaceNicks. Closes #407
+* telegram: Fix possible nil when using channels (telegram). #410
+* telegram: Fix panic (telegram). Closes #410
+* telegram: Handle channel posts correctly
+* mattermost: Update GetFileLinks to API_V4
+
+# v1.9.1
+## New features
+* telegram: Add QuoteDisable option (telegram). Closes #399. See QuoteDisable in matterbridge.toml.sample
+## Enhancements
+* discord: Send mediaserver link to Discord in Webhook mode (discord) (#405)
+* mattermost: Print list of valid team names when team not found (#390)
+* slack: Strip markdown URLs with blank text (slack) (#392)
+## Bugfix
+* slack/mattermost: Make our callbackid more unique. Fixes issue with running multiple matterbridge on the same channel (slack,mattermost)
+* telegram: fix newlines in multiline messages #399
+* telegram: Revert #378
+
+# v1.9.0 (the refactor release)
+## New features
+* general: better debug messages
+* general: better support for environment variables override
+* general: Ability to disable sending join/leave messages to other gateways. #382
+* slack: Allow Slack @usergroups to be parsed as human-friendly names #379
+* slack: Provide better context for shared posts from Slack<=>Slack enhancement #369
+* telegram: Convert nicks automatically into HTML when MessageFormat is set to HTML #378
+* irc: Add DebugLevel option 
+
+## Bugfix
+* slack: Ignore restricted_action on channel join (slack). Closes #387
+* slack: Add slack attachment support to matterhook
+* slack: Update userlist on join (slack). Closes #372
+
+# v1.8.0
+## New features
+* general: Send chat notification if media is too big to be re-uploaded to MediaServer. See #359
+* general: Download (and upload) avatar images from mattermost and telegram when mediaserver is configured. Closes #362
+* general: Add label support in RemoteNickFormat
+* general: Prettier info/debug log output
+* mattermost: Download files and reupload to supported bridges (mattermost). Closes #357
+* slack: Add ShowTopicChange option. Allow/disable topic change messages (currently only from slack). Closes #353
+* slack: Add support for file comments (slack). Closes #346
+* telegram: Add comment to file upload from telegram. Show comments on all bridges. Closes #358
+* telegram: Add markdown support (telegram). #355
+* api: Give api access to whole config.Message (and events). Closes #374
+
+## Bugfix
+* discord: Check for a valid WebhookURL (discord). Closes #367
+* discord: Fix role mention replace issues
+* irc: Truncate messages sent to IRC based on byte count (#368)
+* mattermost: Add file download urls also to mattermost webhooks #356
+* telegram: Fix panic on nil messages (telegram). Closes #366
+* telegram: Fix the UseInsecureURL text (telegram). Closes #184
+
+# v1.7.1
+## Bugfix
+* telegram: Enable Long Polling for Telegram. Reduces bandwidth consumption. (#350)
+
+# v1.7.0
+## New features
+* matrix: Add support for deleting messages from/to matrix (matrix). Closes #320
+* xmpp: Ignore <subject> messages (xmpp). #272
+* irc: Add twitch support (irc) to README / wiki
+
+## Bugfix
+* general: Change RemoteNickFormat replacement order. Closes #336
+* general: Make edits/delete work for bridges that gets reused. Closes #342
+* general: Lowercase irc channels in config. Closes #348
+* matrix: Fix possible panics (matrix). Closes #333
+* matrix: Add an extension to images without one (matrix). #331
+* api: Obey the Gateway value from the json (api). Closes #344
+* xmpp: Print only debug messages when specified (xmpp). Closes #345
+* xmpp: Allow xmpp to receive the extra messages (file uploads) when text is empty. #295
+
+# v1.6.3
+## Bugfix
+* slack: Fix connection issues
+* slack: Add more debug messages
+* irc: Convert received IRC channel names to lowercase. Fixes #329 (#330)
+
+# v1.6.2
+## Bugfix
+* mattermost: Crashes while connecting to Mattermost (regression). Closes #327
+
+# v1.6.1
+## Bugfix
+* general: Display of nicks not longer working (regression). Closes #323
+
+# v1.6.0
+## New features
+* sshchat: New protocol support added (https://github.com/shazow/ssh-chat)
+* general: Allow specifying maximum download size of media using MediaDownloadSize (slack,telegram,matrix)
+* api: Add (simple, one listener) long-polling support (api). Closes #307
+* telegram: Add support for forwarded messages. Closes #313
+* telegram: Add support for Audio/Voice files (telegram). Closes #314
+* irc: Add RejoinDelay option. Delay to rejoin after channel kick (irc). Closes #322
+
+## Bugfix
+* telegram: Also use HTML in edited messages (telegram). Closes #315
+* matrix: Fix panic (matrix). Closes #316
+
+# v1.5.1
+
+## Bugfix
+* irc: Fix irc ACTION regression (irc). Closes #306
+* irc: Split on UTF-8 for MessageSplit (irc). Closes #308
+
+# v1.5.0
+## New features
+* general: remote mediaserver support. See MediaServerDownload and MediaServerUpload in matterbridge.toml.sample
+  more information on https://github.com/42wim/matterbridge/wiki/Mediaserver-setup-%5Badvanced%5D
+* general: Add support for ReplaceNicks using regexp to replace nicks. Closes #269 (see matterbridge.toml.sample)
+* general: Add support for ReplaceMessages using regexp to replace messages. #269 (see matterbridge.toml.sample)
+* irc: Add MessageSplit option to split messages on MessageLength (irc). Closes #281
+* matrix: Add support for uploading images/video (matrix). Closes #302
+* matrix: Add support for uploaded images/video (matrix) 
+
+## Bugfix
+* telegram: Add webp extension to stickers if necessary (telegram)
+* mattermost: Break when re-login fails (mattermost)
+
+# v1.4.1
+## Bugfix
+* telegram: fix issue with uploading for images/documents/stickers
+* slack: remove double messages sent to other bridges when uploading files
+* irc: Fix strict user handling of girc (irc). Closes #298 
+
+# v1.4.0
+## Breaking changes
+* general: `[general]` settings don't override the specific bridge settings
+
+## New features
+* irc: Replace sorcix/irc and go-ircevent with girc, this should be give better reconnects
+* steam: Add support for bridging to individual steam chats. (steam) (#294)
+* telegram: Download files from telegram and reupload to supported bridges (telegram). #278
+* slack: Add support to upload files to slack, from bridges with private urls like slack/mattermost/telegram. (slack)
+* discord: Add support to upload files to discord, from bridges with private urls like slack/mattermost/telegram. (discord)
+* general: Add systemd service file (#291)
+* general: Add support for DEBUG=1 envvar to enable debug. Closes #283
+* general: Add StripNick option, only allow alphanumerical nicks. Closes #285
+
+## Bugfix
+* gitter: Use room.URI instead of room.Name. (gitter) (#293)
+* slack: Allow slack messages with variables (eg. @here) to be formatted correctly. (slack) (#288)
+* slack: Resolve slack channel to human-readable name. (slack) (#282)
+* slack: Use DisplayName instead of deprecated username (slack). Closes #276
+* slack: Allowed Slack bridge to extract simpler link format. (#287)
+* irc: Strip irc colors correct, strip also ctrl chars (irc)
+
+# v1.3.1
+## New features
+* Support mattermost 4.3.0 and every other 4.x as api4 should be stable (mattermost)
+## Bugfix
+* Use bot username if specified (slack). Closes #273
+
+# v1.3.0
+## New features
+* Relay slack_attachments from mattermost to slack (slack). Closes #260
+* Add support for quoting previous message when replying (telegram). #237
+* Add support for Quakenet auth (irc). Closes #263
+* Download files (max size 1MB) from slack and reupload to mattermost (slack/mattermost). Closes #255
+
+## Enhancements
+* Backoff for 60 seconds when reconnecting too fast (irc) #267
+* Use override username if specified (mattermost). #260
+
+## Bugfix
+* Try to not forward slack unfurls. Closes #266
+
+# v1.2.0
+## Breaking changes
+* If you're running a discord bridge, update to this release before 16 october otherwise
+it will stop working. (see https://discordapp.com/developers/docs/reference)
+
+## New features
+* general: Add delete support. (actually delete the messages on bridges that support it)
+    (mattermost,discord,gitter,slack,telegram)
+
+## Bugfix
+* Do not break messages on newline (slack). Closes #258 
+* Update telegram library
+* Update discord library (supports v6 API now). Old API is deprecated on 16 October
+
+# v1.1.2
+## New features
+* general: also build darwin binaries
+* mattermost: add support for mattermost 4.2.x
+
+## Bugfix 
+* mattermost: Send images when text is empty regression. (mattermost). Closes #254
+* slack: also send the first messsage after connect. #252
+
+# v1.1.1
+## Bugfix
+* mattermost: fix public links
+
+# v1.1.0
+## New features
+* general: Add better editing support. (actually edit the messages on bridges that support it)
+	(mattermost,discord,gitter,slack,telegram)
+* mattermost: use API v4 (removes support for mattermost < 3.8)
+* mattermost: add support for personal access tokens (since mattermost 4.1)
+	Use ```Token="yourtoken"``` in mattermost config
+	See https://docs.mattermost.com/developer/personal-access-tokens.html for more info
+* matrix: Relay notices (matrix). Closes #243
+* irc: Add a charset option. Closes #247
+
+## Bugfix
+* slack: Handle leave/join events (slack). Closes #246
+* slack: Replace mentions from other bridges. (slack). Closes #233
+* gitter: remove ZWSP after messages
+
+# v1.0.1
+## New features
+* mattermost: add support for mattermost 4.1.x
+* discord: allow a webhookURL per channel #239
+
+# v1.0.0
+## New features
+* general: Add action support for slack,mattermost,irc,gitter,matrix,xmpp,discord. #199
+* discord: Shows the username instead of the server nickname #234
+
+# v1.0.0-rc1
+## New features
+* general: Add action support for slack,mattermost,irc,gitter,matrix,xmpp,discord. #199
+
+## Bugfix
+* general: Handle same account in multiple gateways better
+* mattermost: ignore edited messages with reactions
+* mattermost: Fix double posting of edited messages by using lru cache
+* irc: update vendor
+
+# v0.16.3
+## Bugfix
+* general: Fix in/out logic. Closes #224 
+* general: Fix message modification
+* slack: Disable message from other bots when using webhooks (slack)
+* mattermost: Return better error messages on mattermost connect
+
+# v0.16.2
+## New features
+* general: binary builds against latest commit are now available on https://bintray.com/42wim/nightly/Matterbridge/_latestVersion
+
+## Bugfix
+* slack: fix loop introduced by relaying message of other bots #219
+* slack: Suppress parent message when child message is received #218
+* mattermost: fix regression when using webhookurl and webhookbindaddress #221
+
+# v0.16.1
+## New features
+* slack: also relay messages of other bots #213
+* mattermost: show also links if public links have not been enabled.
+
+## Bugfix
+* mattermost, slack: fix connecting logic #216
+
+# v0.16.0
+## Breaking Changes
+* URL,UseAPI,BindAddress is deprecated. Your config has to be updated.
+  * URL => WebhookURL
+  * BindAddress => WebhookBindAddress
+  * UseAPI => removed 
+  This change allows you to specify a WebhookURL and a token (slack,discord), so that
+  messages will be sent with the webhook, but received via the token (API)
+  If you have not specified WebhookURL and WebhookBindAddress the API (login or token) 
+  will be used automatically. (no need for UseAPI)
+
+## New features
+* mattermost: add support for mattermost 4.0
+* steam: New protocol support added (http://store.steampowered.com/)
+* discord: Support for embedded messages (sent by other bots)
+  Shows title, description and URL of embedded messages (sent by other bots)
+  To enable add ```ShowEmbeds=true``` to your discord config 
+* discord: ```WebhookURL``` posting support added (thanks @saury07) #204
+  Discord API does not allow to change the name of the user posting, but webhooks does.
+
+## Changes
+* general: all :emoji: will be converted to unicode, providing consistent emojis across all bridges
+* telegram: Add ```UseInsecureURL``` option for telegram (default false)
+  WARNING! If enabled this will relay GIF/stickers/documents and other attachments as URLs
+  Those URLs will contain your bot-token. This may not be what you want.
+  For now there is no secure way to relay GIF/stickers/documents without seeing your token.
+
+## Bugfix
+* irc: detect charset and try to convert it to utf-8 before sending it to other bridges. #209 #210
+* slack: Remove label from URLs (slack). #205
+* slack: Relay <>& correctly to other bridges #215
+* steam: Fix channel id bug in steam (channels are off by 0x18000000000000)
+* general: various improvements
+* general: samechannelgateway now relays messages correct again #207
+
+
+# v0.16.0-rc2
+## Breaking Changes
+* URL,UseAPI,BindAddress is deprecated. Your config has to be updated.
+  * URL => WebhookURL
+  * BindAddress => WebhookBindAddress
+  * UseAPI => removed 
+  This change allows you to specify a WebhookURL and a token (slack,discord), so that
+  messages will be sent with the webhook, but received via the token (API)
+  If you have not specified WebhookURL and WebhookBindAddress the API (login or token) 
+  will be used automatically. (no need for UseAPI)
+
+## Bugfix since rc1
+* steam: Fix channel id bug in steam (channels are off by 0x18000000000000)
+* telegram: Add UseInsecureURL option for telegram (default false)
+  WARNING! If enabled this will relay GIF/stickers/documents and other attachments as URLs
+  Those URLs will contain your bot-token. This may not be what you want.
+  For now there is no secure way to relay GIF/stickers/documents without seeing your token.
+* irc: detect charset and try to convert it to utf-8 before sending it to other bridges. #209 #210
+* general: various improvements
+
+
+# v0.16.0-rc1
+## Breaking Changes
+* URL,UseAPI,BindAddress is deprecated. Your config has to be updated.
+  * URL => WebhookURL
+  * BindAddress => WebhookBindAddress
+  * UseAPI => removed 
+  This change allows you to specify a WebhookURL and a token (slack,discord), so that
+  messages will be sent with the webhook, but received via the token (API)
+  If you have not specified WebhookURL and WebhookBindAddress the API (login or token) 
+  will be used automatically. (no need for UseAPI)
+
+## New features
+* steam: New protocol support added (http://store.steampowered.com/)
+* discord: WebhookURL posting support added (thanks @saury07) #204
+  Discord API does not allow to change the name of the user posting, but webhooks does.
+
+## Bugfix
+* general: samechannelgateway now relays messages correct again #207
+* slack: Remove label from URLs (slack). #205
+
+# v0.15.0
+## New features
+* general: add option IgnoreMessages for all protocols (see mattebridge.toml.sample)
+  Messages matching these regexp will be ignored and not sent to other bridges
+  e.g. IgnoreMessages="^~~ badword"
+* telegram: add support for sticker/video/photo/document #184
+
+## Changes
+* api: add userid to each message #200
+
+## Bugfix
+* discord: fix crash in memberupdate #198
+* mattermost: Fix incorrect behaviour of EditDisable (mattermost). Fixes #197 
+* irc: Do not relay join/part of ourselves (irc). Closes #190 
+* irc: make reconnections more robust. #153
+* gitter: update library, fixes possible crash
+
+# v0.14.0
+## New features
+* api: add token authentication
+* mattermost: add support for mattermost 3.10.0
+
+## Changes
+* api: gateway name is added in JSON messages
+* api: lowercase JSON keys
+* api: channel name isn't needed in config #195
+
+## Bugfix
+* discord: Add hashtag to channelname (when translating from id) (discord)
+* mattermost: Fix a panic. #186
+* mattermost: use teamid cache if possible. Fixes a panic
+* api: post valid json. #185
+* api: allow reuse of api in different gateways. #189
+* general: Fix utf-8 issues for {NOPINGNICK}. #193
+
+# v0.13.0
+## New features
+* irc: Limit message length. ```MessageLength=400```
+  Maximum length of message sent to irc server. If it exceeds <message clipped> will be add to the message.
+* irc: Add NOPINGNICK option. 
+  The string "{NOPINGNICK}" (case sensitive) will be replaced by the actual nick / username, but with a ZWSP inside the nick, so the irc user with the same nick won't get pinged.   
+  See https://github.com/42wim/matterbridge/issues/175 for more information
+
+## Bugfix
+* slack: Fix sending to different channels on same account (slack). Closes #177
+* telegram: Fix incorrect usernames being sent. Closes #181
+
+
+# v0.12.1
+## New features
+* telegram: Add UseFirstName option (telegram). Closes #144
+* matrix: Add NoHomeServerSuffix. Option to disable homeserver on username (matrix). Closes #160.
+
+## Bugfix
+* xmpp: Add Compatibility for Cisco Jabber (xmpp) (#166)
+* irc: Fix JoinChannel argument to use IRC channel key (#172)
+* discord: Fix possible crash on nil (discord)
+* discord: Replace long ids in channel metions (discord). Fixes #174
+
+# v0.12.0
+## Changes
+* general: edited messages are now being sent by default on discord/mattermost/telegram/slack. See "New Features"
+
+## New features
+* general: add support for edited messages. 
+  Add new keyword EditDisable (false/true), default false. Which means by default edited messages will be sent to other bridges.
+  Add new keyword EditSuffix , default "". You can change this eg to "(edited)", this will be appended to every edit message.
+* mattermost: support mattermost v3.9.x
+* general: Add support for HTTP{S}_PROXY env variables (#162)
+* discord: Strip custom emoji metadata (discord). Closes #148
+
+## Bugfix
+* slack: Ignore error on private channel join (slack) Fixes #150 
+* mattermost: fix crash on reconnects when server is down. Closes #163
+* irc: Relay messages starting with ! (irc). Closes #164
+
+# v0.11.0
+## New features
+* general: reusing the same account on multiple gateways now also reuses the connection.
+  This is particuarly useful for irc. See #87
+* general: the Name is now REQUIRED and needs to be UNIQUE for each gateway configuration
+* telegram:  Support edited messages (telegram). See #141
+* mattermost: Add support for showing/hiding join/leave messages from mattermost. Closes #147
+* mattermost: Reconnect on session removal/timeout (mattermost)
+* mattermost: Support mattermost v3.8.x
+* irc:  Rejoin channel when kicked (irc).
+
+## Bugfix
+* mattermost: Remove space after nick (mattermost). Closes #142
+* mattermost: Modify iconurl correctly (mattermost).
+* irc: Fix join/leave regression (irc)
+
+# v0.10.3
+## Bugfix
+* slack: Allow bot tokens for now without warning (slack). Closes #140 (fixes user_is_bot message on channel join)
+
+# v0.10.2
+## New features
+* general: gops agent added. Allows for more debugging. See #134
+* general: toml inline table support added for config file
+
+## Bugfix
+* all: vendored libs updated
+
+## Changes
+* general: add more informative messages on startup
+
 # v0.10.1
 ## Bugfix
 * gitter: Fix sending messages on new channel join.

ci/bintray.sh

@@ -0,0 +1,27 @@
+#!/bin/bash
+go version |grep go1.10 || exit
+VERSION=$(git describe --tags)
+mkdir ci/binaries
+GOOS=windows GOARCH=amd64 go build -ldflags "-s -w -X main.githash=$(git log --pretty=format:'%h' -n 1)" -o ci/binaries/matterbridge-$VERSION-windows-amd64.exe
+GOOS=linux GOARCH=amd64 go build -ldflags "-s -w -X main.githash=$(git log --pretty=format:'%h' -n 1)" -o ci/binaries/matterbridge-$VERSION-linux-amd64
+GOOS=linux GOARCH=arm go build -ldflags "-s -w -X main.githash=$(git log --pretty=format:'%h' -n 1)" -o ci/binaries/matterbridge-$VERSION-linux-arm
+GOOS=darwin GOARCH=amd64 go build -ldflags "-s -w -X main.githash=$(git log --pretty=format:'%h' -n 1)" -o ci/binaries/matterbridge-$VERSION-darwin-amd64
+cd ci
+cat > deploy.json <<EOF
+{
+    "package": {
+        "name": "Matterbridge",
+        "repo": "nightly",
+        "subject": "42wim"
+    },
+    "version": {
+        "name": "$VERSION"
+    },
+    "files":
+        [
+        {"includePattern": "ci/binaries/(.*)", "uploadPattern":"\$1"}
+        ],
+    "publish": true
+}
+EOF
+

contrib/matterbridge.service

@@ -0,0 +1,11 @@
+[Unit]
+Description=matterbridge
+After=network.target
+
+[Service]
+ExecStart=/usr/bin/matterbridge -conf /etc/matterbridge/bridge.toml
+User=matterbridge
+Group=matterbridge
+
+[Install]
+WantedBy=multi-user.target

docker/arm/Dockerfile

@@ -0,0 +1,11 @@
+FROM cmosh/alpine-arm:edge
+ENTRYPOINT ["/bin/matterbridge"]
+
+COPY . /go/src/github.com/42wim/matterbridge
+RUN apk update && apk add go git gcc musl-dev ca-certificates \
+        && cd /go/src/github.com/42wim/matterbridge \
+        && export GOPATH=/go \
+        && go get \
+        && go build -x -ldflags "-X main.githash=$(git log --pretty=format:'%h' -n 1)" -o /bin/matterbridge \
+        && rm -rf /go \
+        && apk del --purge git go gcc musl-dev

gateway/gateway.go

@@ -1,71 +1,108 @@
 package gateway
 
 import (
+	"bytes"
 	"fmt"
+	"io/ioutil"
+	"net/http"
+	"os"
+
 	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/api"
 	"github.com/42wim/matterbridge/bridge/config"
-	log "github.com/Sirupsen/logrus"
-	"reflect"
+	bdiscord "github.com/42wim/matterbridge/bridge/discord"
+	bgitter "github.com/42wim/matterbridge/bridge/gitter"
+	birc "github.com/42wim/matterbridge/bridge/irc"
+	bmatrix "github.com/42wim/matterbridge/bridge/matrix"
+	bmattermost "github.com/42wim/matterbridge/bridge/mattermost"
+	brocketchat "github.com/42wim/matterbridge/bridge/rocketchat"
+	bslack "github.com/42wim/matterbridge/bridge/slack"
+	bsshchat "github.com/42wim/matterbridge/bridge/sshchat"
+	bsteam "github.com/42wim/matterbridge/bridge/steam"
+	btelegram "github.com/42wim/matterbridge/bridge/telegram"
+	bxmpp "github.com/42wim/matterbridge/bridge/xmpp"
+	bzulip "github.com/42wim/matterbridge/bridge/zulip"
+	"github.com/hashicorp/golang-lru"
+	log "github.com/sirupsen/logrus"
+	//	"github.com/davecgh/go-spew/spew"
+	"crypto/sha1"
+	"path/filepath"
+	"regexp"
 	"strings"
 	"time"
+
+	"github.com/peterhellberg/emojilib"
 )
 
 type Gateway struct {
 	*config.Config
-	MyConfig        *config.Gateway
-	Bridges         map[string]*bridge.Bridge
-	ChannelsOut     map[string][]string
-	ChannelsIn      map[string][]string
-	ChannelOptions  map[string]config.ChannelOptions
-	Name            string
-	Message         chan config.Message
-	DestChannelFunc func(msg *config.Message, dest string) []string
+	Router         *Router
+	MyConfig       *config.Gateway
+	Bridges        map[string]*bridge.Bridge
+	Channels       map[string]*config.ChannelInfo
+	ChannelOptions map[string]config.ChannelOptions
+	Message        chan config.Message
+	Name           string
+	Messages       *lru.Cache
 }
 
-func New(cfg *config.Config, gateway *config.Gateway) *Gateway {
-	gw := &Gateway{}
-	gw.Name = gateway.Name
-	gw.Config = cfg
-	gw.MyConfig = gateway
-	gw.Message = make(chan config.Message)
-	gw.Bridges = make(map[string]*bridge.Bridge)
-	gw.DestChannelFunc = gw.getDestChannel
+type BrMsgID struct {
+	br        *bridge.Bridge
+	ID        string
+	ChannelID string
+}
+
+var flog *log.Entry
+
+var bridgeMap = map[string]bridge.Factory{
+	"api":        api.New,
+	"discord":    bdiscord.New,
+	"gitter":     bgitter.New,
+	"irc":        birc.New,
+	"mattermost": bmattermost.New,
+	"matrix":     bmatrix.New,
+	"rocketchat": brocketchat.New,
+	"slack":      bslack.New,
+	"sshchat":    bsshchat.New,
+	"steam":      bsteam.New,
+	"telegram":   btelegram.New,
+	"xmpp":       bxmpp.New,
+	"zulip":      bzulip.New,
+}
+
+func init() {
+	flog = log.WithFields(log.Fields{"prefix": "gateway"})
+}
+
+func New(cfg config.Gateway, r *Router) *Gateway {
+	gw := &Gateway{Channels: make(map[string]*config.ChannelInfo), Message: r.Message,
+		Router: r, Bridges: make(map[string]*bridge.Bridge), Config: r.Config}
+	cache, _ := lru.New(5000)
+	gw.Messages = cache
+	gw.AddConfig(&cfg)
 	return gw
 }
 
 func (gw *Gateway) AddBridge(cfg *config.Bridge) error {
-	for _, br := range gw.Bridges {
-		if br.Account == cfg.Account {
-			return nil
-		}
+	br := gw.Router.getBridge(cfg.Account)
+	if br == nil {
+		br = bridge.New(cfg)
+		br.Config = gw.Router.Config
+		br.General = &gw.General
+		// set logging
+		br.Log = log.WithFields(log.Fields{"prefix": "bridge"})
+		brconfig := &bridge.Config{Remote: gw.Message, Log: log.WithFields(log.Fields{"prefix": br.Protocol}), Bridge: br}
+		// add the actual bridger for this protocol to this bridge using the bridgeMap
+		br.Bridger = bridgeMap[br.Protocol](brconfig)
 	}
-	log.Infof("Starting bridge: %s ", cfg.Account)
-	br := bridge.New(gw.Config, cfg, gw.Message)
-	gw.mapChannelsToBridge(br, gw.ChannelsOut)
-	gw.mapChannelsToBridge(br, gw.ChannelsIn)
+	gw.mapChannelsToBridge(br)
 	gw.Bridges[cfg.Account] = br
-	err := br.Connect()
-	if err != nil {
-		return fmt.Errorf("Bridge %s failed to start: %v", br.Account, err)
-	}
-	err = br.JoinChannels()
-	if err != nil {
-		return fmt.Errorf("Bridge %s failed to join channel: %v", br.Account, err)
-	}
 	return nil
 }
 
-func (gw *Gateway) mapChannelsToBridge(br *bridge.Bridge, cMap map[string][]string) {
-	for _, channel := range cMap[br.Account] {
-		if _, ok := gw.ChannelOptions[br.Account+channel]; ok {
-			br.ChannelsOut[channel] = gw.ChannelOptions[br.Account+channel]
-		} else {
-			br.ChannelsOut[channel] = config.ChannelOptions{}
-		}
-	}
-}
-
-func (gw *Gateway) Start() error {
+func (gw *Gateway) AddConfig(cfg *config.Gateway) error {
+	gw.Name = cfg.Name
+	gw.MyConfig = cfg
 	gw.mapChannels()
 	for _, br := range append(gw.MyConfig.In, append(gw.MyConfig.InOut, gw.MyConfig.Out...)...) {
 		err := gw.AddBridge(&br)
@@ -73,27 +110,13 @@ func (gw *Gateway) Start() error {
 			return err
 		}
 	}
-	go gw.handleReceive()
 	return nil
 }
 
-func (gw *Gateway) handleReceive() {
-	for {
-		select {
-		case msg := <-gw.Message:
-			if msg.Event == config.EVENT_FAILURE {
-				for _, br := range gw.Bridges {
-					if msg.Account == br.Account {
-						go gw.reconnectBridge(br)
-					}
-				}
-			}
-			if !gw.ignoreMessage(&msg) {
-				msg.Timestamp = time.Now()
-				for _, br := range gw.Bridges {
-					gw.handleMessage(msg, br)
-				}
-			}
+func (gw *Gateway) mapChannelsToBridge(br *bridge.Bridge) {
+	for ID, channel := range gw.Channels {
+		if br.Account == channel.Account {
+			br.Channels[ID] = *channel
 		}
 	}
 }
@@ -102,122 +125,384 @@ func (gw *Gateway) reconnectBridge(br *bridge.Bridge) {
 	br.Disconnect()
 	time.Sleep(time.Second * 5)
 RECONNECT:
-	log.Infof("Reconnecting %s", br.Account)
+	flog.Infof("Reconnecting %s", br.Account)
 	err := br.Connect()
 	if err != nil {
-		log.Errorf("Reconnection failed: %s. Trying again in 60 seconds", err)
+		flog.Errorf("Reconnection failed: %s. Trying again in 60 seconds", err)
 		time.Sleep(time.Second * 60)
 		goto RECONNECT
 	}
+	br.Joined = make(map[string]bool)
 	br.JoinChannels()
 }
 
+func (gw *Gateway) mapChannelConfig(cfg []config.Bridge, direction string) {
+	for _, br := range cfg {
+		if isApi(br.Account) {
+			br.Channel = "api"
+		}
+		// make sure to lowercase irc channels in config #348
+		if strings.HasPrefix(br.Account, "irc.") {
+			br.Channel = strings.ToLower(br.Channel)
+		}
+		ID := br.Channel + br.Account
+		if _, ok := gw.Channels[ID]; !ok {
+			channel := &config.ChannelInfo{Name: br.Channel, Direction: direction, ID: ID, Options: br.Options, Account: br.Account,
+				SameChannel: make(map[string]bool)}
+			channel.SameChannel[gw.Name] = br.SameChannel
+			gw.Channels[channel.ID] = channel
+		} else {
+			// if we already have a key and it's not our current direction it means we have a bidirectional inout
+			if gw.Channels[ID].Direction != direction {
+				gw.Channels[ID].Direction = "inout"
+			}
+		}
+		gw.Channels[ID].SameChannel[gw.Name] = br.SameChannel
+	}
+}
+
 func (gw *Gateway) mapChannels() error {
-	options := make(map[string]config.ChannelOptions)
-	m := make(map[string][]string)
-	for _, br := range gw.MyConfig.Out {
-		m[br.Account] = append(m[br.Account], br.Channel)
-		options[br.Account+br.Channel] = br.Options
-	}
-	gw.ChannelsOut = m
-	m = nil
-	m = make(map[string][]string)
-	for _, br := range gw.MyConfig.In {
-		m[br.Account] = append(m[br.Account], br.Channel)
-		options[br.Account+br.Channel] = br.Options
-	}
-	gw.ChannelsIn = m
-	for _, br := range gw.MyConfig.InOut {
-		gw.ChannelsIn[br.Account] = append(gw.ChannelsIn[br.Account], br.Channel)
-		gw.ChannelsOut[br.Account] = append(gw.ChannelsOut[br.Account], br.Channel)
-		options[br.Account+br.Channel] = br.Options
-	}
-	gw.ChannelOptions = options
+	gw.mapChannelConfig(gw.MyConfig.In, "in")
+	gw.mapChannelConfig(gw.MyConfig.Out, "out")
+	gw.mapChannelConfig(gw.MyConfig.InOut, "inout")
 	return nil
 }
 
-func (gw *Gateway) getDestChannel(msg *config.Message, dest string) []string {
-	channels := gw.ChannelsIn[msg.Account]
-	// broadcast to every out channel (irc QUIT)
-	if msg.Event == config.EVENT_JOIN_LEAVE && msg.Channel == "" {
-		return gw.ChannelsOut[dest]
-	}
-	for _, channel := range channels {
-		if channel == msg.Channel {
-			return gw.ChannelsOut[dest]
-		}
-	}
-	return []string{}
-}
+func (gw *Gateway) getDestChannel(msg *config.Message, dest bridge.Bridge) []config.ChannelInfo {
+	var channels []config.ChannelInfo
 
-func (gw *Gateway) handleMessage(msg config.Message, dest *bridge.Bridge) {
-	// only relay join/part when configged
-	if msg.Event == config.EVENT_JOIN_LEAVE && !gw.Bridges[dest.Account].Config.ShowJoinPart {
-		return
+	// for messages received from the api check that the gateway is the specified one
+	if msg.Protocol == "api" && gw.Name != msg.Gateway {
+		return channels
 	}
-	originchannel := msg.Channel
-	channels := gw.DestChannelFunc(&msg, dest.Account)
-	for _, channel := range channels {
-		// do not send the message to the bridge we come from if also the channel is the same
-		if msg.Account == dest.Account && channel == originchannel {
+
+	// if source channel is in only, do nothing
+	for _, channel := range gw.Channels {
+		// lookup the channel from the message
+		if channel.ID == getChannelID(*msg) {
+			// we only have destinations if the original message is from an "in" (sending) channel
+			if !strings.Contains(channel.Direction, "in") {
+				return channels
+			}
 			continue
 		}
-		msg.Channel = channel
-		if msg.Channel == "" {
-			log.Debug("empty channel")
-			return
+	}
+	for _, channel := range gw.Channels {
+		if _, ok := gw.Channels[getChannelID(*msg)]; !ok {
+			continue
+		}
+
+		// do samechannelgateway flogic
+		if channel.SameChannel[msg.Gateway] {
+			if msg.Channel == channel.Name && msg.Account != dest.Account {
+				channels = append(channels, *channel)
+			}
+			continue
+		}
+		if strings.Contains(channel.Direction, "out") && channel.Account == dest.Account && gw.validGatewayDest(msg, channel) {
+			channels = append(channels, *channel)
+		}
+	}
+	return channels
+}
+
+func (gw *Gateway) handleMessage(msg config.Message, dest *bridge.Bridge) []*BrMsgID {
+	var brMsgIDs []*BrMsgID
+
+	// if we have an attached file, or other info
+	if msg.Extra != nil {
+		if len(msg.Extra[config.EVENT_FILE_FAILURE_SIZE]) != 0 {
+			if msg.Text == "" {
+				return brMsgIDs
+			}
+		}
+	}
+
+	// Avatar downloads are only relevant for telegram and mattermost for now
+	if msg.Event == config.EVENT_AVATAR_DOWNLOAD {
+		if dest.Protocol != "mattermost" &&
+			dest.Protocol != "telegram" {
+			return brMsgIDs
+		}
+	}
+
+	// only relay join/part when configured
+	if msg.Event == config.EVENT_JOIN_LEAVE && !gw.Bridges[dest.Account].GetBool("ShowJoinPart") {
+		return brMsgIDs
+	}
+
+	// only relay topic change when configured
+	if msg.Event == config.EVENT_TOPIC_CHANGE && !gw.Bridges[dest.Account].GetBool("ShowTopicChange") {
+		return brMsgIDs
+	}
+
+	// broadcast to every out channel (irc QUIT)
+	if msg.Channel == "" && msg.Event != config.EVENT_JOIN_LEAVE {
+		flog.Debug("empty channel")
+		return brMsgIDs
+	}
+
+	originchannel := msg.Channel
+	origmsg := msg
+	channels := gw.getDestChannel(&msg, *dest)
+	for _, channel := range channels {
+		// Only send the avatar download event to ourselves.
+		if msg.Event == config.EVENT_AVATAR_DOWNLOAD {
+			if channel.ID != getChannelID(origmsg) {
+				continue
+			}
+		} else {
+			// do not send to ourself for any other event
+			if channel.ID == getChannelID(origmsg) {
+				continue
+			}
+		}
+		flog.Debugf("=> Sending %#v from %s (%s) to %s (%s)", msg, msg.Account, originchannel, dest.Account, channel.Name)
+		msg.Channel = channel.Name
+		msg.Avatar = gw.modifyAvatar(origmsg, dest)
+		msg.Username = gw.modifyUsername(origmsg, dest)
+		msg.ID = ""
+		if res, ok := gw.Messages.Get(origmsg.ID); ok {
+			IDs := res.([]*BrMsgID)
+			for _, id := range IDs {
+				// check protocol, bridge name and channelname
+				// for people that reuse the same bridge multiple times. see #342
+				if dest.Protocol == id.br.Protocol && dest.Name == id.br.Name && channel.ID == id.ChannelID {
+					msg.ID = id.ID
+				}
+			}
 		}
-		log.Debugf("Sending %#v from %s (%s) to %s (%s)", msg, msg.Account, originchannel, dest.Account, channel)
-		gw.modifyUsername(&msg, dest)
 		// for api we need originchannel as channel
 		if dest.Protocol == "api" {
 			msg.Channel = originchannel
 		}
-		err := dest.Send(msg)
+		mID, err := dest.Send(msg)
 		if err != nil {
-			fmt.Println(err)
+			flog.Error(err)
+		}
+		// append the message ID (mID) from this bridge (dest) to our brMsgIDs slice
+		if mID != "" {
+			flog.Debugf("mID %s: %s", dest.Account, mID)
+			brMsgIDs = append(brMsgIDs, &BrMsgID{dest, mID, channel.ID})
 		}
 	}
+	return brMsgIDs
 }
 
 func (gw *Gateway) ignoreMessage(msg *config.Message) bool {
-	if msg.Text == "" {
-		log.Debugf("ignoring empty message %#v from %s", msg, msg.Account)
+	// if we don't have the bridge, ignore it
+	if _, ok := gw.Bridges[msg.Account]; !ok {
 		return true
 	}
-	for _, entry := range strings.Fields(gw.Bridges[msg.Account].Config.IgnoreNicks) {
+
+	// check if we need to ignore a empty message
+	if msg.Text == "" {
+		// we have an attachment or actual bytes, do not ignore
+		if msg.Extra != nil &&
+			(msg.Extra["attachments"] != nil ||
+				len(msg.Extra["file"]) > 0 ||
+				len(msg.Extra[config.EVENT_FILE_FAILURE_SIZE]) > 0) {
+			return false
+		}
+		flog.Debugf("ignoring empty message %#v from %s", msg, msg.Account)
+		return true
+	}
+
+	// is the username in IgnoreNicks field
+	for _, entry := range strings.Fields(gw.Bridges[msg.Account].GetString("IgnoreNicks")) {
 		if msg.Username == entry {
-			log.Debugf("ignoring %s from %s", msg.Username, msg.Account)
+			flog.Debugf("ignoring %s from %s", msg.Username, msg.Account)
 			return true
 		}
 	}
+
+	// does the message match regex in IgnoreMessages field
+	// TODO do not compile regexps everytime
+	for _, entry := range strings.Fields(gw.Bridges[msg.Account].GetString("IgnoreMessages")) {
+		if entry != "" {
+			re, err := regexp.Compile(entry)
+			if err != nil {
+				flog.Errorf("incorrect regexp %s for %s", entry, msg.Account)
+				continue
+			}
+			if re.MatchString(msg.Text) {
+				flog.Debugf("matching %s. ignoring %s from %s", entry, msg.Text, msg.Account)
+				return true
+			}
+		}
+	}
 	return false
 }
 
-func (gw *Gateway) modifyMessage(msg *config.Message, dest *bridge.Bridge) {
-	val := reflect.ValueOf(gw.Config).Elem()
-	for i := 0; i < val.NumField(); i++ {
-		typeField := val.Type().Field(i)
-		// look for the protocol map (both lowercase)
-		if strings.ToLower(typeField.Name) == dest.Protocol {
-			// get the Protocol struct from the map
-			protoCfg := val.Field(i).MapIndex(reflect.ValueOf(dest.Name))
-			//config.SetNickFormat(msg, protoCfg.Interface().(config.Protocol))
-			val.Field(i).SetMapIndex(reflect.ValueOf(dest.Name), protoCfg)
+func (gw *Gateway) modifyUsername(msg config.Message, dest *bridge.Bridge) string {
+	br := gw.Bridges[msg.Account]
+	msg.Protocol = br.Protocol
+	if gw.Config.General.StripNick || dest.GetBool("StripNick") {
+		re := regexp.MustCompile("[^a-zA-Z0-9]+")
+		msg.Username = re.ReplaceAllString(msg.Username, "")
+	}
+	nick := dest.GetString("RemoteNickFormat")
+	if nick == "" {
+		nick = gw.Config.General.RemoteNickFormat
+	}
+
+	// loop to replace nicks
+	for _, outer := range br.GetStringSlice2D("ReplaceNicks") {
+		search := outer[0]
+		replace := outer[1]
+		// TODO move compile to bridge init somewhere
+		re, err := regexp.Compile(search)
+		if err != nil {
+			flog.Errorf("regexp in %s failed: %s", msg.Account, err)
 			break
 		}
+		msg.Username = re.ReplaceAllString(msg.Username, replace)
+	}
+
+	if len(msg.Username) > 0 {
+		// fix utf-8 issue #193
+		i := 0
+		for index := range msg.Username {
+			if i == 1 {
+				i = index
+				break
+			}
+			i++
+		}
+		nick = strings.Replace(nick, "{NOPINGNICK}", msg.Username[:i]+"​"+msg.Username[i:], -1)
+	}
+
+	nick = strings.Replace(nick, "{BRIDGE}", br.Name, -1)
+	nick = strings.Replace(nick, "{PROTOCOL}", br.Protocol, -1)
+	nick = strings.Replace(nick, "{LABEL}", br.GetString("Label"), -1)
+	nick = strings.Replace(nick, "{NICK}", msg.Username, -1)
+	return nick
+}
+
+func (gw *Gateway) modifyAvatar(msg config.Message, dest *bridge.Bridge) string {
+	iconurl := gw.Config.General.IconURL
+	if iconurl == "" {
+		iconurl = dest.GetString("IconURL")
+	}
+	iconurl = strings.Replace(iconurl, "{NICK}", msg.Username, -1)
+	if msg.Avatar == "" {
+		msg.Avatar = iconurl
+	}
+	return msg.Avatar
+}
+
+func (gw *Gateway) modifyMessage(msg *config.Message) {
+	// replace :emoji: to unicode
+	msg.Text = emojilib.Replace(msg.Text)
+
+	br := gw.Bridges[msg.Account]
+	// loop to replace messages
+	for _, outer := range br.GetStringSlice2D("ReplaceMessages") {
+		search := outer[0]
+		replace := outer[1]
+		// TODO move compile to bridge init somewhere
+		re, err := regexp.Compile(search)
+		if err != nil {
+			flog.Errorf("regexp in %s failed: %s", msg.Account, err)
+			break
+		}
+		msg.Text = re.ReplaceAllString(msg.Text, replace)
+	}
+
+	// messages from api have Gateway specified, don't overwrite
+	if msg.Protocol != "api" {
+		msg.Gateway = gw.Name
 	}
 }
 
-func (gw *Gateway) modifyUsername(msg *config.Message, dest *bridge.Bridge) {
-	br := gw.Bridges[msg.Account]
-	msg.Protocol = br.Protocol
-	nick := gw.Config.General.RemoteNickFormat
-	if nick == "" {
-		nick = dest.Config.RemoteNickFormat
+// handleFiles uploads or places all files on the given msg to the MediaServer and
+// adds the new URL of the file on the MediaServer onto the given msg.
+func (gw *Gateway) handleFiles(msg *config.Message) {
+	reg := regexp.MustCompile("[^a-zA-Z0-9]+")
+
+	// If we don't have a attachfield or we don't have a mediaserver configured return
+	if msg.Extra == nil || (gw.Config.General.MediaServerUpload == "" && gw.Config.General.MediaDownloadPath == "") {
+		return
+	}
+
+	// If we don't have files, nothing to upload.
+	if len(msg.Extra["file"]) == 0 {
+		return
+	}
+
+	client := &http.Client{
+		Timeout: time.Second * 5,
+	}
+
+	for i, f := range msg.Extra["file"] {
+		fi := f.(config.FileInfo)
+		ext := filepath.Ext(fi.Name)
+		fi.Name = fi.Name[0 : len(fi.Name)-len(ext)]
+		fi.Name = reg.ReplaceAllString(fi.Name, "_")
+		fi.Name = fi.Name + ext
+
+		sha1sum := fmt.Sprintf("%x", sha1.Sum(*fi.Data))[:8]
+
+		if gw.Config.General.MediaServerUpload != "" {
+			// Use MediaServerUpload. Upload using a PUT HTTP request and basicauth.
+
+			url := gw.Config.General.MediaServerUpload + "/" + sha1sum + "/" + fi.Name
+
+			req, err := http.NewRequest("PUT", url, bytes.NewReader(*fi.Data))
+			if err != nil {
+				flog.Errorf("mediaserver upload failed, could not create request: %#v", err)
+				continue
+			}
+
+			flog.Debugf("mediaserver upload url: %s", url)
+
+			req.Header.Set("Content-Type", "binary/octet-stream")
+			_, err = client.Do(req)
+			if err != nil {
+				flog.Errorf("mediaserver upload failed, could not Do request: %#v", err)
+				continue
+			}
+		} else {
+			// Use MediaServerPath. Place the file on the current filesystem.
+
+			dir := gw.Config.General.MediaDownloadPath + "/" + sha1sum
+			err := os.Mkdir(dir, os.ModePerm)
+			if err != nil && !os.IsExist(err) {
+				flog.Errorf("mediaserver path failed, could not mkdir: %s %#v", err, err)
+				continue
+			}
+
+			path := dir + "/" + fi.Name
+			flog.Debugf("mediaserver path placing file: %s", path)
+
+			err = ioutil.WriteFile(path, *fi.Data, os.ModePerm)
+			if err != nil {
+				flog.Errorf("mediaserver path failed, could not writefile: %s %#v", err, err)
+				continue
+			}
+		}
+
+		// Download URL.
+		durl := gw.Config.General.MediaServerDownload + "/" + sha1sum + "/" + fi.Name
+
+		flog.Debugf("mediaserver download URL = %s", durl)
+
+		// We uploaded/placed the file successfully. Add the SHA and URL.
+		extra := msg.Extra["file"][i].(config.FileInfo)
+		extra.URL = durl
+		extra.SHA = sha1sum
+		msg.Extra["file"][i] = extra
 	}
-	nick = strings.Replace(nick, "{NICK}", msg.Username, -1)
-	nick = strings.Replace(nick, "{BRIDGE}", br.Name, -1)
-	nick = strings.Replace(nick, "{PROTOCOL}", br.Protocol, -1)
-	msg.Username = nick
+}
+
+func (gw *Gateway) validGatewayDest(msg *config.Message, channel *config.ChannelInfo) bool {
+	return msg.Gateway == gw.Name
+}
+
+func getChannelID(msg config.Message) string {
+	return msg.Channel + msg.Account
+}
+
+func isApi(account string) bool {
+	return strings.HasPrefix(account, "api.")
 }

gateway/gateway_test.go

@@ -0,0 +1,279 @@
+package gateway
+
+import (
+	"fmt"
+	"strconv"
+
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/stretchr/testify/assert"
+
+	"testing"
+)
+
+var testconfig = []byte(`
+[irc.freenode]
+[mattermost.test]
+[gitter.42wim]
+[discord.test]
+[slack.test]
+
+[[gateway]]
+    name = "bridge1"
+    enable=true
+    
+    [[gateway.inout]]
+    account = "irc.freenode"
+    channel = "#wimtesting"
+    
+    [[gateway.inout]]
+    account="gitter.42wim"
+    channel="42wim/testroom"
+    #channel="matterbridge/Lobby"
+
+    [[gateway.inout]]
+    account = "discord.test"
+    channel = "general"
+    
+    [[gateway.inout]]
+    account="slack.test"
+    channel="testing"
+	`)
+
+var testconfig2 = []byte(`
+[irc.freenode]
+[mattermost.test]
+[gitter.42wim]
+[discord.test]
+[slack.test]
+
+[[gateway]]
+    name = "bridge1"
+    enable=true
+    
+    [[gateway.in]]
+    account = "irc.freenode"
+    channel = "#wimtesting"
+    
+    [[gateway.in]]
+    account="gitter.42wim"
+    channel="42wim/testroom"
+
+    [[gateway.inout]]
+    account = "discord.test"
+    channel = "general"
+    
+    [[gateway.out]]
+    account="slack.test"
+    channel="testing"
+[[gateway]]
+    name = "bridge2"
+    enable=true
+    
+    [[gateway.in]]
+    account = "irc.freenode"
+    channel = "#wimtesting2"
+    
+    [[gateway.out]]
+    account="gitter.42wim"
+    channel="42wim/testroom"
+
+    [[gateway.out]]
+    account = "discord.test"
+    channel = "general2"
+	`)
+
+var testconfig3 = []byte(`
+[irc.zzz]
+[telegram.zzz]
+[slack.zzz]
+[[gateway]]
+name="bridge"
+enable=true
+
+    [[gateway.inout]]
+    account="irc.zzz"
+    channel="#main"		
+
+    [[gateway.inout]]
+    account="telegram.zzz"
+    channel="-1111111111111"
+
+    [[gateway.inout]]
+    account="slack.zzz"
+    channel="irc"	
+	
+[[gateway]]
+name="announcements"
+enable=true
+	
+    [[gateway.in]]
+    account="telegram.zzz"
+    channel="-2222222222222"	
+	
+    [[gateway.out]]
+    account="irc.zzz"
+    channel="#main"		
+	
+    [[gateway.out]]
+    account="irc.zzz"
+    channel="#main-help"	
+
+    [[gateway.out]]
+    account="telegram.zzz"
+    channel="--333333333333"	
+
+    [[gateway.out]]
+    account="slack.zzz"
+    channel="general"		
+	
+[[gateway]]
+name="bridge2"
+enable=true
+
+    [[gateway.inout]]
+    account="irc.zzz"
+    channel="#main-help"	
+
+    [[gateway.inout]]
+    account="telegram.zzz"
+    channel="--444444444444"	
+
+	
+[[gateway]]
+name="bridge3"
+enable=true
+
+    [[gateway.inout]]
+    account="irc.zzz"
+    channel="#main-telegram"	
+
+    [[gateway.inout]]
+    account="telegram.zzz"
+    channel="--333333333333"
+`)
+
+func maketestRouter(input []byte) *Router {
+	cfg := config.NewConfigFromString(input)
+	r, err := NewRouter(cfg)
+	if err != nil {
+		fmt.Println(err)
+	}
+	return r
+}
+func TestNewRouter(t *testing.T) {
+	r := maketestRouter(testconfig)
+	assert.Equal(t, 1, len(r.Gateways))
+	assert.Equal(t, 4, len(r.Gateways["bridge1"].Bridges))
+	assert.Equal(t, 4, len(r.Gateways["bridge1"].Channels))
+
+	r = maketestRouter(testconfig2)
+	assert.Equal(t, 2, len(r.Gateways))
+	assert.Equal(t, 4, len(r.Gateways["bridge1"].Bridges))
+	assert.Equal(t, 3, len(r.Gateways["bridge2"].Bridges))
+	assert.Equal(t, 4, len(r.Gateways["bridge1"].Channels))
+	assert.Equal(t, 3, len(r.Gateways["bridge2"].Channels))
+	assert.Equal(t, &config.ChannelInfo{Name: "42wim/testroom", Direction: "out",
+		ID: "42wim/testroomgitter.42wim", Account: "gitter.42wim",
+		SameChannel: map[string]bool{"bridge2": false}},
+		r.Gateways["bridge2"].Channels["42wim/testroomgitter.42wim"])
+	assert.Equal(t, &config.ChannelInfo{Name: "42wim/testroom", Direction: "in",
+		ID: "42wim/testroomgitter.42wim", Account: "gitter.42wim",
+		SameChannel: map[string]bool{"bridge1": false}},
+		r.Gateways["bridge1"].Channels["42wim/testroomgitter.42wim"])
+	assert.Equal(t, &config.ChannelInfo{Name: "general", Direction: "inout",
+		ID: "generaldiscord.test", Account: "discord.test",
+		SameChannel: map[string]bool{"bridge1": false}},
+		r.Gateways["bridge1"].Channels["generaldiscord.test"])
+}
+
+func TestGetDestChannel(t *testing.T) {
+	r := maketestRouter(testconfig2)
+	msg := &config.Message{Text: "test", Channel: "general", Account: "discord.test", Gateway: "bridge1", Protocol: "discord", Username: "test"}
+	for _, br := range r.Gateways["bridge1"].Bridges {
+		switch br.Account {
+		case "discord.test":
+			assert.Equal(t, []config.ChannelInfo{{Name: "general", Account: "discord.test", Direction: "inout", ID: "generaldiscord.test", SameChannel: map[string]bool{"bridge1": false}, Options: config.ChannelOptions{Key: ""}}},
+				r.Gateways["bridge1"].getDestChannel(msg, *br))
+		case "slack.test":
+			assert.Equal(t, []config.ChannelInfo{{Name: "testing", Account: "slack.test", Direction: "out", ID: "testingslack.test", SameChannel: map[string]bool{"bridge1": false}, Options: config.ChannelOptions{Key: ""}}},
+				r.Gateways["bridge1"].getDestChannel(msg, *br))
+		case "gitter.42wim":
+			assert.Equal(t, []config.ChannelInfo(nil), r.Gateways["bridge1"].getDestChannel(msg, *br))
+		case "irc.freenode":
+			assert.Equal(t, []config.ChannelInfo(nil), r.Gateways["bridge1"].getDestChannel(msg, *br))
+		}
+	}
+}
+
+func TestGetDestChannelAdvanced(t *testing.T) {
+	r := maketestRouter(testconfig3)
+	var msgs []*config.Message
+	i := 0
+	for _, gw := range r.Gateways {
+		for _, channel := range gw.Channels {
+			msgs = append(msgs, &config.Message{Text: "text" + strconv.Itoa(i), Channel: channel.Name, Account: channel.Account, Gateway: gw.Name, Username: "user" + strconv.Itoa(i)})
+			i++
+		}
+	}
+	hits := make(map[string]int)
+	for _, gw := range r.Gateways {
+		for _, br := range gw.Bridges {
+			for _, msg := range msgs {
+				channels := gw.getDestChannel(msg, *br)
+				if gw.Name != msg.Gateway {
+					assert.Equal(t, []config.ChannelInfo(nil), channels)
+					continue
+				}
+				switch gw.Name {
+				case "bridge":
+					if (msg.Channel == "#main" || msg.Channel == "-1111111111111" || msg.Channel == "irc") && (msg.Account == "irc.zzz" || msg.Account == "telegram.zzz" || msg.Account == "slack.zzz") {
+						hits[gw.Name]++
+						switch br.Account {
+						case "irc.zzz":
+							assert.Equal(t, []config.ChannelInfo{{Name: "#main", Account: "irc.zzz", Direction: "inout", ID: "#mainirc.zzz", SameChannel: map[string]bool{"bridge": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+						case "telegram.zzz":
+							assert.Equal(t, []config.ChannelInfo{{Name: "-1111111111111", Account: "telegram.zzz", Direction: "inout", ID: "-1111111111111telegram.zzz", SameChannel: map[string]bool{"bridge": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+						case "slack.zzz":
+							assert.Equal(t, []config.ChannelInfo{{Name: "irc", Account: "slack.zzz", Direction: "inout", ID: "ircslack.zzz", SameChannel: map[string]bool{"bridge": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+						}
+					}
+				case "bridge2":
+					if (msg.Channel == "#main-help" || msg.Channel == "--444444444444") && (msg.Account == "irc.zzz" || msg.Account == "telegram.zzz") {
+						hits[gw.Name]++
+						switch br.Account {
+						case "irc.zzz":
+							assert.Equal(t, []config.ChannelInfo{{Name: "#main-help", Account: "irc.zzz", Direction: "inout", ID: "#main-helpirc.zzz", SameChannel: map[string]bool{"bridge2": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+						case "telegram.zzz":
+							assert.Equal(t, []config.ChannelInfo{{Name: "--444444444444", Account: "telegram.zzz", Direction: "inout", ID: "--444444444444telegram.zzz", SameChannel: map[string]bool{"bridge2": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+						}
+					}
+				case "bridge3":
+					if (msg.Channel == "#main-telegram" || msg.Channel == "--333333333333") && (msg.Account == "irc.zzz" || msg.Account == "telegram.zzz") {
+						hits[gw.Name]++
+						switch br.Account {
+						case "irc.zzz":
+							assert.Equal(t, []config.ChannelInfo{{Name: "#main-telegram", Account: "irc.zzz", Direction: "inout", ID: "#main-telegramirc.zzz", SameChannel: map[string]bool{"bridge3": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+						case "telegram.zzz":
+							assert.Equal(t, []config.ChannelInfo{{Name: "--333333333333", Account: "telegram.zzz", Direction: "inout", ID: "--333333333333telegram.zzz", SameChannel: map[string]bool{"bridge3": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+						}
+					}
+				case "announcements":
+					if msg.Channel != "-2222222222222" && msg.Account != "telegram" {
+						assert.Equal(t, []config.ChannelInfo(nil), channels)
+						continue
+					}
+					hits[gw.Name]++
+					switch br.Account {
+					case "irc.zzz":
+						assert.Equal(t, []config.ChannelInfo{{Name: "#main", Account: "irc.zzz", Direction: "out", ID: "#mainirc.zzz", SameChannel: map[string]bool{"announcements": false}, Options: config.ChannelOptions{Key: ""}}, {Name: "#main-help", Account: "irc.zzz", Direction: "out", ID: "#main-helpirc.zzz", SameChannel: map[string]bool{"announcements": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+					case "slack.zzz":
+						assert.Equal(t, []config.ChannelInfo{{Name: "general", Account: "slack.zzz", Direction: "out", ID: "generalslack.zzz", SameChannel: map[string]bool{"announcements": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+					case "telegram.zzz":
+						assert.Equal(t, []config.ChannelInfo{{Name: "--333333333333", Account: "telegram.zzz", Direction: "out", ID: "--333333333333telegram.zzz", SameChannel: map[string]bool{"announcements": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+					}
+				}
+			}
+		}
+	}
+	assert.Equal(t, map[string]int{"bridge3": 4, "bridge": 9, "announcements": 3, "bridge2": 4}, hits)
+}

gateway/router.go

@@ -0,0 +1,111 @@
+package gateway
+
+import (
+	"fmt"
+
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	samechannelgateway "github.com/42wim/matterbridge/gateway/samechannel"
+	//	"github.com/davecgh/go-spew/spew"
+	"time"
+)
+
+type Router struct {
+	Gateways map[string]*Gateway
+	Message  chan config.Message
+	*config.Config
+}
+
+func NewRouter(cfg *config.Config) (*Router, error) {
+	r := &Router{Message: make(chan config.Message), Gateways: make(map[string]*Gateway), Config: cfg}
+	sgw := samechannelgateway.New(cfg)
+	gwconfigs := sgw.GetConfig()
+
+	for _, entry := range append(gwconfigs, cfg.Gateway...) {
+		if !entry.Enable {
+			continue
+		}
+		if entry.Name == "" {
+			return nil, fmt.Errorf("%s", "Gateway without name found")
+		}
+		if _, ok := r.Gateways[entry.Name]; ok {
+			return nil, fmt.Errorf("Gateway with name %s already exists", entry.Name)
+		}
+		r.Gateways[entry.Name] = New(entry, r)
+	}
+	return r, nil
+}
+
+func (r *Router) Start() error {
+	m := make(map[string]*bridge.Bridge)
+	for _, gw := range r.Gateways {
+		flog.Infof("Parsing gateway %s", gw.Name)
+		for _, br := range gw.Bridges {
+			m[br.Account] = br
+		}
+	}
+	for _, br := range m {
+		flog.Infof("Starting bridge: %s ", br.Account)
+		err := br.Connect()
+		if err != nil {
+			return fmt.Errorf("Bridge %s failed to start: %v", br.Account, err)
+		}
+		err = br.JoinChannels()
+		if err != nil {
+			return fmt.Errorf("Bridge %s failed to join channel: %v", br.Account, err)
+		}
+	}
+	go r.handleReceive()
+	return nil
+}
+
+func (r *Router) getBridge(account string) *bridge.Bridge {
+	for _, gw := range r.Gateways {
+		if br, ok := gw.Bridges[account]; ok {
+			return br
+		}
+	}
+	return nil
+}
+
+func (r *Router) handleReceive() {
+	for msg := range r.Message {
+		if msg.Event == config.EVENT_FAILURE {
+		Loop:
+			for _, gw := range r.Gateways {
+				for _, br := range gw.Bridges {
+					if msg.Account == br.Account {
+						go gw.reconnectBridge(br)
+						break Loop
+					}
+				}
+			}
+		}
+		if msg.Event == config.EVENT_REJOIN_CHANNELS {
+			for _, gw := range r.Gateways {
+				for _, br := range gw.Bridges {
+					if msg.Account == br.Account {
+						br.Joined = make(map[string]bool)
+						br.JoinChannels()
+					}
+				}
+			}
+		}
+		for _, gw := range r.Gateways {
+			// record all the message ID's of the different bridges
+			var msgIDs []*BrMsgID
+			if !gw.ignoreMessage(&msg) {
+				msg.Timestamp = time.Now()
+				gw.modifyMessage(&msg)
+				gw.handleFiles(&msg)
+				for _, br := range gw.Bridges {
+					msgIDs = append(msgIDs, gw.handleMessage(msg, br)...)
+				}
+				// only add the message ID if it doesn't already exists
+				if _, ok := gw.Messages.Get(msg.ID); !ok && msg.ID != "" {
+					gw.Messages.Add(msg.ID, msgIDs)
+				}
+			}
+		}
+	}
+}

gateway/samechannel/samechannel.go

@@ -2,48 +2,27 @@ package samechannelgateway
 
 import (
 	"github.com/42wim/matterbridge/bridge/config"
-	"github.com/42wim/matterbridge/gateway"
 )
 
 type SameChannelGateway struct {
 	*config.Config
-	MyConfig *config.SameChannelGateway
-	Channels []string
-	Name     string
 }
 
-func New(cfg *config.Config, gatewayCfg *config.SameChannelGateway) *SameChannelGateway {
-	return &SameChannelGateway{
-		MyConfig: gatewayCfg,
-		Channels: gatewayCfg.Channels,
-		Name:     gatewayCfg.Name,
-		Config:   cfg}
+func New(cfg *config.Config) *SameChannelGateway {
+	return &SameChannelGateway{Config: cfg}
 }
 
-func (sgw *SameChannelGateway) Start() error {
-	gw := gateway.New(sgw.Config, &config.Gateway{Name: sgw.Name})
-	gw.DestChannelFunc = sgw.getDestChannel
-	for _, account := range sgw.MyConfig.Accounts {
-		for _, channel := range sgw.Channels {
-			br := config.Bridge{Account: account, Channel: channel}
-			gw.MyConfig.InOut = append(gw.MyConfig.InOut, br)
+func (sgw *SameChannelGateway) GetConfig() []config.Gateway {
+	var gwconfigs []config.Gateway
+	cfg := sgw.Config
+	for _, gw := range cfg.SameChannelGateway {
+		gwconfig := config.Gateway{Name: gw.Name, Enable: gw.Enable}
+		for _, account := range gw.Accounts {
+			for _, channel := range gw.Channels {
+				gwconfig.InOut = append(gwconfig.InOut, config.Bridge{Account: account, Channel: channel, SameChannel: true})
+			}
 		}
+		gwconfigs = append(gwconfigs, gwconfig)
 	}
-	return gw.Start()
-}
-
-func (sgw *SameChannelGateway) validChannel(channel string) bool {
-	for _, c := range sgw.Channels {
-		if c == channel {
-			return true
-		}
-	}
-	return false
-}
-
-func (sgw *SameChannelGateway) getDestChannel(msg *config.Message, dest string) []string {
-	if sgw.validChannel(msg.Channel) {
-		return []string{msg.Channel}
-	}
-	return []string{}
+	return gwconfigs
 }

gateway/samechannel/samechannel_test.go

@@ -0,0 +1,32 @@
+package samechannelgateway
+
+import (
+	"fmt"
+
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/BurntSushi/toml"
+	"github.com/stretchr/testify/assert"
+
+	"testing"
+)
+
+var testconfig = `
+[mattermost.test]
+[slack.test]
+
+[[samechannelgateway]]
+   enable = true
+   name = "blah"
+      accounts = [ "mattermost.test","slack.test" ]
+      channels = [ "testing","testing2","testing10"]
+`
+
+func TestGetConfig(t *testing.T) {
+	var cfg *config.Config
+	if _, err := toml.Decode(testconfig, &cfg); err != nil {
+		fmt.Println(err)
+	}
+	sgw := New(cfg)
+	configs := sgw.GetConfig()
+	assert.Equal(t, []config.Gateway{{Name: "blah", Enable: true, In: []config.Bridge(nil), Out: []config.Bridge(nil), InOut: []config.Bridge{{Account: "mattermost.test", Channel: "testing", Options: config.ChannelOptions{Key: ""}, SameChannel: true}, {Account: "mattermost.test", Channel: "testing2", Options: config.ChannelOptions{Key: ""}, SameChannel: true}, {Account: "mattermost.test", Channel: "testing10", Options: config.ChannelOptions{Key: ""}, SameChannel: true}, {Account: "slack.test", Channel: "testing", Options: config.ChannelOptions{Key: ""}, SameChannel: true}, {Account: "slack.test", Channel: "testing2", Options: config.ChannelOptions{Key: ""}, SameChannel: true}, {Account: "slack.test", Channel: "testing10", Options: config.ChannelOptions{Key: ""}, SameChannel: true}}}}, configs)
+}

hook/rockethook/rockethook.go

@@ -99,10 +99,9 @@ func (c *Client) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 
 // Receive returns an incoming message from mattermost outgoing webhooks URL.
 func (c *Client) Receive() Message {
-	for {
-		select {
-		case msg := <-c.In:
-			return msg
-		}
+	var msg Message
+	for msg = range c.In {
+		return msg
 	}
+	return msg
 }

matterbridge.go

@@ -3,23 +3,24 @@ package main
 import (
 	"flag"
 	"fmt"
+	"os"
+	"strings"
+
 	"github.com/42wim/matterbridge/bridge/config"
 	"github.com/42wim/matterbridge/gateway"
-	"github.com/42wim/matterbridge/gateway/samechannel"
-	log "github.com/Sirupsen/logrus"
 	"github.com/google/gops/agent"
+	log "github.com/sirupsen/logrus"
+	prefixed "github.com/x-cray/logrus-prefixed-formatter"
 )
 
 var (
-	version = "0.10.2-dev"
+	version = "1.11.0"
 	githash string
 )
 
-func init() {
-	log.SetFormatter(&log.TextFormatter{FullTimestamp: true})
-}
-
 func main() {
+	log.SetFormatter(&prefixed.TextFormatter{PrefixPadding: 13, DisableColors: true, FullTimestamp: true})
+	flog := log.WithFields(log.Fields{"prefix": "main"})
 	flagConfig := flag.String("conf", "matterbridge.toml", "config file")
 	flagDebug := flag.Bool("debug", false, "enable debug")
 	flagVersion := flag.Bool("version", false, "show version")
@@ -33,38 +34,25 @@ func main() {
 		fmt.Printf("version: %s %s\n", version, githash)
 		return
 	}
-	flag.Parse()
-	if *flagDebug {
-		log.Info("Enabling debug")
+	if *flagDebug || os.Getenv("DEBUG") == "1" {
+		log.SetFormatter(&prefixed.TextFormatter{PrefixPadding: 13, DisableColors: true, FullTimestamp: false, ForceFormatting: true})
+		flog.Info("Enabling debug")
 		log.SetLevel(log.DebugLevel)
 	}
-	log.Printf("Running version %s %s", version, githash)
+	flog.Printf("Running version %s %s", version, githash)
+	if strings.Contains(version, "-dev") {
+		flog.Println("WARNING: THIS IS A DEVELOPMENT VERSION. Things may break.")
+	}
 	cfg := config.NewConfig(*flagConfig)
-	for _, gw := range cfg.SameChannelGateway {
-		if !gw.Enable {
-			continue
-		}
-		log.Printf("Starting samechannel gateway %#v", gw.Name)
-		g := samechannelgateway.New(cfg, &gw)
-		err := g.Start()
-		if err != nil {
-			log.Fatalf("Starting gateway failed %#v", err)
-		}
-		log.Printf("Started samechannel gateway %#v", gw.Name)
+	cfg.General.Debug = *flagDebug
+	r, err := gateway.NewRouter(cfg)
+	if err != nil {
+		flog.Fatalf("Starting gateway failed: %s", err)
 	}
-
-	for _, gw := range cfg.Gateway {
-		if !gw.Enable {
-			continue
-		}
-		log.Printf("Starting gateway %#v", gw.Name)
-		g := gateway.New(cfg, &gw)
-		err := g.Start()
-		if err != nil {
-			log.Fatalf("Starting gateway failed %#v", err)
-		}
-		log.Printf("Started gateway %#v", gw.Name)
+	err = r.Start()
+	if err != nil {
+		flog.Fatalf("Starting gateway failed: %s", err)
 	}
-	log.Printf("Gateway(s) started succesfully. Now relaying messages")
+	flog.Printf("Gateway(s) started succesfully. Now relaying messages")
 	select {}
 }

matterbridge.toml.simple

@@ -1,3 +1,4 @@
+#WARNING: as this file contains credentials, be sure to set correct file permissions
 [irc]
     [irc.freenode]
     Server="irc.freenode.net:6667"
@@ -5,8 +6,8 @@
 
 [mattermost]
     [mattermost.work]
-    useAPI=true
-    Server="yourmattermostserver.domain"
+    #do not prefix it wit http:// or https://
+    Server="yourmattermostserver.domain" 
     Team="yourteam"
     Login="yourlogin"
     Password="yourpass"
@@ -15,18 +16,19 @@
 [[gateway]]
 name="gateway1"
 enable=true
-    [[gateway.in]]
+    [[gateway.inout]]
     account="irc.freenode"
     channel="#testing"
 
-    [[gateway.out]]
-    account="irc.freenode"
-    channel="#testing"
-
-    [[gateway.in]]
+    [[gateway.inout]]
     account="mattermost.work"
     channel="off-topic"
     
-    [[gateway.out]]
-    account="mattermost.work"
-    channel="off-topic"
+#simpler config possible since v0.10.2
+#[[gateway]]
+#name="gateway2"
+#enable=true
+#inout = [
+#    { account="irc.freenode", channel="#testing", options={key="channelkey"}},
+#    { account="mattermost.work", channel="off-topic" },
+#]

matterclient/matterclient.go

@@ -1,9 +1,11 @@
 package matterclient
 
 import (
+	"crypto/md5"
 	"crypto/tls"
 	"encoding/json"
 	"errors"
+	"fmt"
 	"net/http"
 	"net/http/cookiejar"
 	"net/url"
@@ -11,9 +13,11 @@ import (
 	"sync"
 	"time"
 
-	log "github.com/Sirupsen/logrus"
+	log "github.com/sirupsen/logrus"
+	prefixed "github.com/x-cray/logrus-prefixed-formatter"
 
 	"github.com/gorilla/websocket"
+	"github.com/hashicorp/golang-lru"
 	"github.com/jpillora/backoff"
 	"github.com/mattermost/platform/model"
 )
@@ -34,42 +38,52 @@ type Message struct {
 	Channel  string
 	Username string
 	Text     string
+	Type     string
+	UserID   string
 }
 
 type Team struct {
 	Team         *model.Team
 	Id           string
-	Channels     *model.ChannelList
-	MoreChannels *model.ChannelList
+	Channels     []*model.Channel
+	MoreChannels []*model.Channel
 	Users        map[string]*model.User
 }
 
 type MMClient struct {
 	sync.RWMutex
 	*Credentials
-	Team        *Team
-	OtherTeams  []*Team
-	Client      *model.Client
-	User        *model.User
-	Users       map[string]*model.User
-	MessageChan chan *Message
-	log         *log.Entry
-	WsClient    *websocket.Conn
-	WsQuit      bool
-	WsAway      bool
-	WsConnected bool
-	WsSequence  int64
-	WsPingChan  chan *model.WebSocketResponse
+	Team          *Team
+	OtherTeams    []*Team
+	Client        *model.Client4
+	User          *model.User
+	Users         map[string]*model.User
+	MessageChan   chan *Message
+	log           *log.Entry
+	WsClient      *websocket.Conn
+	WsQuit        bool
+	WsAway        bool
+	WsConnected   bool
+	WsSequence    int64
+	WsPingChan    chan *model.WebSocketResponse
+	ServerVersion string
+	OnWsConnect   func()
+	lruCache      *lru.Cache
 }
 
 func New(login, pass, team, server string) *MMClient {
 	cred := &Credentials{Login: login, Pass: pass, Team: team, Server: server}
 	mmclient := &MMClient{Credentials: cred, MessageChan: make(chan *Message, 100), Users: make(map[string]*model.User)}
-	mmclient.log = log.WithFields(log.Fields{"module": "matterclient"})
-	log.SetFormatter(&log.TextFormatter{FullTimestamp: true})
+	log.SetFormatter(&prefixed.TextFormatter{PrefixPadding: 13, DisableColors: true})
+	mmclient.log = log.WithFields(log.Fields{"prefix": "matterclient"})
+	mmclient.lruCache, _ = lru.New(500)
 	return mmclient
 }
 
+func (m *MMClient) SetDebugLog() {
+	log.SetFormatter(&prefixed.TextFormatter{PrefixPadding: 13, DisableColors: true, FullTimestamp: false, ForceFormatting: true})
+}
+
 func (m *MMClient) SetLogLevel(level string) {
 	l, err := log.ParseLevel(level)
 	if err != nil {
@@ -82,7 +96,7 @@ func (m *MMClient) SetLogLevel(level string) {
 func (m *MMClient) Login() error {
 	// check if this is a first connect or a reconnection
 	firstConnection := true
-	if m.WsConnected == true {
+	if m.WsConnected {
 		firstConnection = false
 	}
 	m.WsConnected = false
@@ -95,39 +109,62 @@ func (m *MMClient) Login() error {
 		Jitter: true,
 	}
 	uriScheme := "https://"
-	wsScheme := "wss://"
 	if m.NoTLS {
 		uriScheme = "http://"
-		wsScheme = "ws://"
 	}
 	// login to mattermost
-	m.Client = model.NewClient(uriScheme + m.Credentials.Server)
-	m.Client.HttpClient.Transport = &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: m.SkipTLSVerify}}
+	m.Client = model.NewAPIv4Client(uriScheme + m.Credentials.Server)
+	m.Client.HttpClient.Transport = &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: m.SkipTLSVerify}, Proxy: http.ProxyFromEnvironment}
 	m.Client.HttpClient.Timeout = time.Second * 10
-	var myinfo *model.Result
+
+	for {
+		d := b.Duration()
+		// bogus call to get the serverversion
+		_, resp := m.Client.Logout()
+		if resp.Error != nil {
+			return fmt.Errorf("%#v", resp.Error.Error())
+		}
+		if firstConnection && !supportedVersion(resp.ServerVersion) {
+			return fmt.Errorf("unsupported mattermost version: %s", resp.ServerVersion)
+		}
+		m.ServerVersion = resp.ServerVersion
+		if m.ServerVersion == "" {
+			m.log.Debugf("Server not up yet, reconnecting in %s", d)
+			time.Sleep(d)
+		} else {
+			m.log.Infof("Found version %s", m.ServerVersion)
+			break
+		}
+	}
+	b.Reset()
+
+	var resp *model.Response
+	//var myinfo *model.Result
 	var appErr *model.AppError
 	var logmsg = "trying login"
 	for {
 		m.log.Debugf("%s %s %s %s", logmsg, m.Credentials.Team, m.Credentials.Login, m.Credentials.Server)
 		if strings.Contains(m.Credentials.Pass, model.SESSION_COOKIE_TOKEN) {
-			m.log.Debugf(logmsg+" with %s", model.SESSION_COOKIE_TOKEN)
+			m.log.Debugf(logmsg + " with token")
 			token := strings.Split(m.Credentials.Pass, model.SESSION_COOKIE_TOKEN+"=")
 			if len(token) != 2 {
 				return errors.New("incorrect MMAUTHTOKEN. valid input is MMAUTHTOKEN=yourtoken")
 			}
 			m.Client.HttpClient.Jar = m.createCookieJar(token[1])
-			m.Client.MockSession(token[1])
-			myinfo, appErr = m.Client.GetMe("")
-			if appErr != nil {
-				return errors.New(appErr.DetailedError)
+			m.Client.AuthToken = token[1]
+			m.Client.AuthType = model.HEADER_BEARER
+			m.User, resp = m.Client.GetMe("")
+			if resp.Error != nil {
+				return resp.Error
 			}
-			if myinfo.Data.(*model.User) == nil {
+			if m.User == nil {
 				m.log.Errorf("LOGIN TOKEN: %s is invalid", m.Credentials.Pass)
 				return errors.New("invalid " + model.SESSION_COOKIE_TOKEN)
 			}
 		} else {
-			myinfo, appErr = m.Client.Login(m.Credentials.Login, m.Credentials.Pass)
+			m.User, resp = m.Client.Login(m.Credentials.Login, m.Credentials.Pass)
 		}
+		appErr = resp.Error
 		if appErr != nil {
 			d := b.Duration()
 			m.log.Debug(appErr.DetailedError)
@@ -153,19 +190,40 @@ func (m *MMClient) Login() error {
 	}
 
 	if m.Team == nil {
-		return errors.New("team not found")
+		validTeamNames := make([]string, len(m.OtherTeams))
+		for i, t := range m.OtherTeams {
+			validTeamNames[i] = t.Team.Name
+		}
+		return fmt.Errorf("Team '%s' not found in %v", m.Credentials.Team, validTeamNames)
+	}
+
+	m.wsConnect()
+
+	return nil
+}
+
+func (m *MMClient) wsConnect() {
+	b := &backoff.Backoff{
+		Min:    time.Second,
+		Max:    5 * time.Minute,
+		Jitter: true,
+	}
+
+	m.WsConnected = false
+	wsScheme := "wss://"
+	if m.NoTLS {
+		wsScheme = "ws://"
 	}
-	// set our team id as default route
-	m.Client.SetTeamId(m.Team.Id)
 
 	// setup websocket connection
-	wsurl := wsScheme + m.Credentials.Server + model.API_URL_SUFFIX + "/users/websocket"
+	wsurl := wsScheme + m.Credentials.Server + model.API_URL_SUFFIX_V4 + "/websocket"
 	header := http.Header{}
 	header.Set(model.HEADER_AUTH, "BEARER "+m.Client.AuthToken)
 
-	m.log.Debug("WsClient: making connection")
+	m.log.Debugf("WsClient: making connection: %s", wsurl)
 	for {
 		wsDialer := &websocket.Dialer{Proxy: http.ProxyFromEnvironment, TLSClientConfig: &tls.Config{InsecureSkipVerify: m.SkipTLSVerify}}
+		var err error
 		m.WsClient, _, err = wsDialer.Dial(wsurl, header)
 		if err != nil {
 			d := b.Duration()
@@ -175,14 +233,12 @@ func (m *MMClient) Login() error {
 		}
 		break
 	}
-	b.Reset()
 
+	m.log.Debug("WsClient: connected")
 	m.WsSequence = 1
 	m.WsPingChan = make(chan *model.WebSocketResponse)
 	// only start to parse WS messages when login is completely done
 	m.WsConnected = true
-
-	return nil
 }
 
 func (m *MMClient) Logout() error {
@@ -190,9 +246,13 @@ func (m *MMClient) Logout() error {
 	m.WsQuit = true
 	m.WsClient.Close()
 	m.WsClient.UnderlyingConn().Close()
-	_, err := m.Client.Logout()
-	if err != nil {
-		return err
+	if strings.Contains(m.Credentials.Pass, model.SESSION_COOKIE_TOKEN) {
+		m.log.Debug("Not invalidating session in logout, credential is a token")
+		return nil
+	}
+	_, resp := m.Client.Logout()
+	if resp.Error != nil {
+		return resp.Error
 	}
 	return nil
 }
@@ -215,21 +275,31 @@ func (m *MMClient) WsReceiver() {
 		if _, rawMsg, err = m.WsClient.ReadMessage(); err != nil {
 			m.log.Error("error:", err)
 			// reconnect
-			m.Login()
+			m.wsConnect()
 		}
 
 		var event model.WebSocketEvent
 		if err := json.Unmarshal(rawMsg, &event); err == nil && event.IsValid() {
-			m.log.Debugf("WsReceiver: %#v", event)
+			m.log.Debugf("WsReceiver event: %#v", event)
 			msg := &Message{Raw: &event, Team: m.Credentials.Team}
 			m.parseMessage(msg)
-			m.MessageChan <- msg
+			// check if we didn't empty the message
+			if msg.Text != "" {
+				m.MessageChan <- msg
+				continue
+			}
+			// if we have file attached but the message is empty, also send it
+			if msg.Post != nil {
+				if msg.Text != "" || len(msg.Post.FileIds) > 0 || msg.Post.Type == "slack_attachment" {
+					m.MessageChan <- msg
+				}
+			}
 			continue
 		}
 
 		var response model.WebSocketResponse
 		if err := json.Unmarshal(rawMsg, &response); err == nil && response.IsValid() {
-			m.log.Debugf("WsReceiver: %#v", response)
+			m.log.Debugf("WsReceiver response: %#v", response)
 			m.parseResponse(response)
 			continue
 		}
@@ -238,7 +308,7 @@ func (m *MMClient) WsReceiver() {
 
 func (m *MMClient) parseMessage(rmsg *Message) {
 	switch rmsg.Raw.Event {
-	case model.WEBSOCKET_EVENT_POSTED:
+	case model.WEBSOCKET_EVENT_POSTED, model.WEBSOCKET_EVENT_POST_EDITED, model.WEBSOCKET_EVENT_POST_DELETED:
 		m.parseActionPost(rmsg)
 		/*
 			case model.ACTION_USER_REMOVED:
@@ -259,46 +329,70 @@ func (m *MMClient) parseResponse(rmsg model.WebSocketResponse) {
 }
 
 func (m *MMClient) parseActionPost(rmsg *Message) {
+	// add post to cache, if it already exists don't relay this again.
+	// this should fix reposts
+	if ok, _ := m.lruCache.ContainsOrAdd(digestString(rmsg.Raw.Data["post"].(string)), true); ok {
+		m.log.Debugf("message %#v in cache, not processing again", rmsg.Raw.Data["post"].(string))
+		rmsg.Text = ""
+		return
+	}
 	data := model.PostFromJson(strings.NewReader(rmsg.Raw.Data["post"].(string)))
 	// we don't have the user, refresh the userlist
 	if m.GetUser(data.UserId) == nil {
-		m.UpdateUsers()
+		m.log.Infof("User %s is not known, ignoring message %s", data)
+		return
 	}
-	rmsg.Username = m.GetUser(data.UserId).Username
+	rmsg.Username = m.GetUserName(data.UserId)
 	rmsg.Channel = m.GetChannelName(data.ChannelId)
-	rmsg.Team = m.GetTeamName(rmsg.Raw.Data["team_id"].(string))
+	rmsg.UserID = data.UserId
+	rmsg.Type = data.Type
+	teamid, _ := rmsg.Raw.Data["team_id"].(string)
+	// edit messsages have no team_id for some reason
+	if teamid == "" {
+		// we can find the team_id from the channelid
+		teamid = m.GetChannelTeamId(data.ChannelId)
+		rmsg.Raw.Data["team_id"] = teamid
+	}
+	if teamid != "" {
+		rmsg.Team = m.GetTeamName(teamid)
+	}
 	// direct message
 	if rmsg.Raw.Data["channel_type"] == "D" {
 		rmsg.Channel = m.GetUser(data.UserId).Username
 	}
 	rmsg.Text = data.Message
 	rmsg.Post = data
-	return
 }
 
 func (m *MMClient) UpdateUsers() error {
-	mmusers, err := m.Client.GetProfiles(0, 50000, "")
-	if err != nil {
-		return errors.New(err.DetailedError)
+	mmusers, resp := m.Client.GetUsers(0, 50000, "")
+	if resp.Error != nil {
+		return errors.New(resp.Error.DetailedError)
 	}
 	m.Lock()
-	m.Users = mmusers.Data.(map[string]*model.User)
+	for _, user := range mmusers {
+		m.Users[user.Id] = user
+	}
 	m.Unlock()
 	return nil
 }
 
 func (m *MMClient) UpdateChannels() error {
-	mmchannels, err := m.Client.GetChannels("")
-	if err != nil {
-		return errors.New(err.DetailedError)
-	}
-	mmchannels2, err := m.Client.GetMoreChannels("")
-	if err != nil {
-		return errors.New(err.DetailedError)
+	mmchannels, resp := m.Client.GetChannelsForTeamForUser(m.Team.Id, m.User.Id, "")
+	if resp.Error != nil {
+		return errors.New(resp.Error.DetailedError)
 	}
 	m.Lock()
-	m.Team.Channels = mmchannels.Data.(*model.ChannelList)
-	m.Team.MoreChannels = mmchannels2.Data.(*model.ChannelList)
+	m.Team.Channels = mmchannels
+	m.Unlock()
+
+	mmchannels, resp = m.Client.GetPublicChannelsForTeam(m.Team.Id, 0, 5000, "")
+	if resp.Error != nil {
+		return errors.New(resp.Error.DetailedError)
+	}
+
+	m.Lock()
+	m.Team.MoreChannels = mmchannels
 	m.Unlock()
 	return nil
 }
@@ -307,9 +401,21 @@ func (m *MMClient) GetChannelName(channelId string) string {
 	m.RLock()
 	defer m.RUnlock()
 	for _, t := range m.OtherTeams {
-		for _, channel := range append(*t.Channels, *t.MoreChannels...) {
-			if channel.Id == channelId {
-				return channel.Name
+		if t == nil {
+			continue
+		}
+		if t.Channels != nil {
+			for _, channel := range t.Channels {
+				if channel.Id == channelId {
+					return channel.Name
+				}
+			}
+		}
+		if t.MoreChannels != nil {
+			for _, channel := range t.MoreChannels {
+				if channel.Id == channelId {
+					return channel.Name
+				}
 			}
 		}
 	}
@@ -324,7 +430,7 @@ func (m *MMClient) GetChannelId(name string, teamId string) string {
 	}
 	for _, t := range m.OtherTeams {
 		if t.Id == teamId {
-			for _, channel := range append(*t.Channels, *t.MoreChannels...) {
+			for _, channel := range append(t.Channels, t.MoreChannels...) {
 				if channel.Name == name {
 					return channel.Id
 				}
@@ -334,11 +440,24 @@ func (m *MMClient) GetChannelId(name string, teamId string) string {
 	return ""
 }
 
+func (m *MMClient) GetChannelTeamId(id string) string {
+	m.RLock()
+	defer m.RUnlock()
+	for _, t := range append(m.OtherTeams, m.Team) {
+		for _, channel := range append(t.Channels, t.MoreChannels...) {
+			if channel.Id == id {
+				return channel.TeamId
+			}
+		}
+	}
+	return ""
+}
+
 func (m *MMClient) GetChannelHeader(channelId string) string {
 	m.RLock()
 	defer m.RUnlock()
 	for _, t := range m.OtherTeams {
-		for _, channel := range append(*t.Channels, *t.MoreChannels...) {
+		for _, channel := range append(t.Channels, t.MoreChannels...) {
 			if channel.Id == channelId {
 				return channel.Header
 			}
@@ -348,55 +467,85 @@ func (m *MMClient) GetChannelHeader(channelId string) string {
 	return ""
 }
 
-func (m *MMClient) PostMessage(channelId string, text string) {
+func (m *MMClient) PostMessage(channelId string, text string) (string, error) {
 	post := &model.Post{ChannelId: channelId, Message: text}
-	m.Client.CreatePost(post)
+	res, resp := m.Client.CreatePost(post)
+	if resp.Error != nil {
+		return "", resp.Error
+	}
+	return res.Id, nil
+}
+
+func (m *MMClient) PostMessageWithFiles(channelId string, text string, fileIds []string) (string, error) {
+	post := &model.Post{ChannelId: channelId, Message: text, FileIds: fileIds}
+	res, resp := m.Client.CreatePost(post)
+	if resp.Error != nil {
+		return "", resp.Error
+	}
+	return res.Id, nil
+}
+
+func (m *MMClient) EditMessage(postId string, text string) (string, error) {
+	post := &model.Post{Message: text}
+	res, resp := m.Client.UpdatePost(postId, post)
+	if resp.Error != nil {
+		return "", resp.Error
+	}
+	return res.Id, nil
+}
+
+func (m *MMClient) DeleteMessage(postId string) error {
+	_, resp := m.Client.DeletePost(postId)
+	if resp.Error != nil {
+		return resp.Error
+	}
+	return nil
 }
 
 func (m *MMClient) JoinChannel(channelId string) error {
 	m.RLock()
 	defer m.RUnlock()
-	for _, c := range *m.Team.Channels {
+	for _, c := range m.Team.Channels {
 		if c.Id == channelId {
 			m.log.Debug("Not joining ", channelId, " already joined.")
 			return nil
 		}
 	}
 	m.log.Debug("Joining ", channelId)
-	_, err := m.Client.JoinChannel(channelId)
-	if err != nil {
-		return errors.New("failed to join")
+	_, resp := m.Client.AddChannelMember(channelId, m.User.Id)
+	if resp.Error != nil {
+		return resp.Error
 	}
 	return nil
 }
 
 func (m *MMClient) GetPostsSince(channelId string, time int64) *model.PostList {
-	res, err := m.Client.GetPostsSince(channelId, time)
-	if err != nil {
+	res, resp := m.Client.GetPostsSince(channelId, time)
+	if resp.Error != nil {
 		return nil
 	}
-	return res.Data.(*model.PostList)
+	return res
 }
 
 func (m *MMClient) SearchPosts(query string) *model.PostList {
-	res, err := m.Client.SearchPosts(query, false)
-	if err != nil {
+	res, resp := m.Client.SearchPosts(m.Team.Id, query, false)
+	if resp.Error != nil {
 		return nil
 	}
-	return res.Data.(*model.PostList)
+	return res
 }
 
 func (m *MMClient) GetPosts(channelId string, limit int) *model.PostList {
-	res, err := m.Client.GetPosts(channelId, 0, limit, "")
-	if err != nil {
+	res, resp := m.Client.GetPostsForChannel(channelId, 0, limit, "")
+	if resp.Error != nil {
 		return nil
 	}
-	return res.Data.(*model.PostList)
+	return res
 }
 
 func (m *MMClient) GetPublicLink(filename string) string {
-	res, err := m.Client.GetPublicLink(filename)
-	if err != nil {
+	res, resp := m.Client.GetFileLink(filename)
+	if resp.Error != nil {
 		return ""
 	}
 	return res
@@ -405,8 +554,27 @@ func (m *MMClient) GetPublicLink(filename string) string {
 func (m *MMClient) GetPublicLinks(filenames []string) []string {
 	var output []string
 	for _, f := range filenames {
-		res, err := m.Client.GetPublicLink(f)
-		if err != nil {
+		res, resp := m.Client.GetFileLink(f)
+		if resp.Error != nil {
+			continue
+		}
+		output = append(output, res)
+	}
+	return output
+}
+
+func (m *MMClient) GetFileLinks(filenames []string) []string {
+	uriScheme := "https://"
+	if m.NoTLS {
+		uriScheme = "http://"
+	}
+
+	var output []string
+	for _, f := range filenames {
+		res, resp := m.Client.GetFileLink(f)
+		if resp.Error != nil {
+			// public links is probably disabled, create the link ourselves
+			output = append(output, uriScheme+m.Credentials.Server+model.API_URL_SUFFIX_V4+"/files/"+f)
 			continue
 		}
 		output = append(output, res)
@@ -415,36 +583,43 @@ func (m *MMClient) GetPublicLinks(filenames []string) []string {
 }
 
 func (m *MMClient) UpdateChannelHeader(channelId string, header string) {
-	data := make(map[string]string)
-	data["channel_id"] = channelId
-	data["channel_header"] = header
+	channel := &model.Channel{Id: channelId, Header: header}
 	m.log.Debugf("updating channelheader %#v, %#v", channelId, header)
-	_, err := m.Client.UpdateChannelHeader(data)
-	if err != nil {
-		log.Error(err)
+	_, resp := m.Client.UpdateChannel(channel)
+	if resp.Error != nil {
+		log.Error(resp.Error)
 	}
 }
 
 func (m *MMClient) UpdateLastViewed(channelId string) {
 	m.log.Debugf("posting lastview %#v", channelId)
-	_, err := m.Client.UpdateLastViewedAt(channelId, true)
-	if err != nil {
-		m.log.Error(err)
+	view := &model.ChannelView{ChannelId: channelId}
+	_, resp := m.Client.ViewChannel(m.User.Id, view)
+	if resp.Error != nil {
+		m.log.Errorf("ChannelView update for %s failed: %s", channelId, resp.Error)
 	}
 }
 
+func (m *MMClient) UpdateUserNick(nick string) error {
+	user := m.User
+	user.Nickname = nick
+	_, resp := m.Client.UpdateUser(user)
+	if resp.Error != nil {
+		return resp.Error
+	}
+	return nil
+}
+
 func (m *MMClient) UsernamesInChannel(channelId string) []string {
-	res, err := m.Client.GetMyChannelMembers()
-	if err != nil {
-		m.log.Errorf("UsernamesInChannel(%s) failed: %s", channelId, err)
+	res, resp := m.Client.GetChannelMembers(channelId, 0, 50000, "")
+	if resp.Error != nil {
+		m.log.Errorf("UsernamesInChannel(%s) failed: %s", channelId, resp.Error)
 		return []string{}
 	}
-	members := res.Data.(*model.ChannelMembers)
+	allusers := m.GetUsers()
 	result := []string{}
-	for _, channel := range *members {
-		if channel.ChannelId == channelId {
-			result = append(result, m.GetUser(channel.UserId).Username)
-		}
+	for _, member := range *res {
+		result = append(result, allusers[member.UserId].Nickname)
 	}
 	return result
 }
@@ -468,22 +643,15 @@ func (m *MMClient) createCookieJar(token string) *cookiejar.Jar {
 func (m *MMClient) SendDirectMessage(toUserId string, msg string) {
 	m.log.Debugf("SendDirectMessage to %s, msg %s", toUserId, msg)
 	// create DM channel (only happens on first message)
-	_, err := m.Client.CreateDirectChannel(toUserId)
-	if err != nil {
-		m.log.Debugf("SendDirectMessage to %#v failed: %s", toUserId, err)
+	_, resp := m.Client.CreateDirectChannel(m.User.Id, toUserId)
+	if resp.Error != nil {
+		m.log.Debugf("SendDirectMessage to %#v failed: %s", toUserId, resp.Error)
 		return
 	}
 	channelName := model.GetDMNameFromIds(toUserId, m.User.Id)
 
 	// update our channels
-	mmchannels, err := m.Client.GetChannels("")
-	if err != nil {
-		m.log.Debug("SendDirectMessage: Couldn't update channels")
-		return
-	}
-	m.Lock()
-	m.Team.Channels = mmchannels.Data.(*model.ChannelList)
-	m.Unlock()
+	m.UpdateChannels()
 
 	// build & send the message
 	msg = strings.Replace(msg, "\r", "", -1)
@@ -509,10 +677,10 @@ func (m *MMClient) GetChannels() []*model.Channel {
 	defer m.RUnlock()
 	var channels []*model.Channel
 	// our primary team channels first
-	channels = append(channels, *m.Team.Channels...)
+	channels = append(channels, m.Team.Channels...)
 	for _, t := range m.OtherTeams {
 		if t.Id != m.Team.Id {
-			channels = append(channels, *t.Channels...)
+			channels = append(channels, t.Channels...)
 		}
 	}
 	return channels
@@ -524,7 +692,7 @@ func (m *MMClient) GetMoreChannels() []*model.Channel {
 	defer m.RUnlock()
 	var channels []*model.Channel
 	for _, t := range m.OtherTeams {
-		channels = append(channels, *t.MoreChannels...)
+		channels = append(channels, t.MoreChannels...)
 	}
 	return channels
 }
@@ -535,8 +703,10 @@ func (m *MMClient) GetTeamFromChannel(channelId string) string {
 	defer m.RUnlock()
 	var channels []*model.Channel
 	for _, t := range m.OtherTeams {
-		channels = append(channels, *t.Channels...)
-		channels = append(channels, *t.MoreChannels...)
+		channels = append(channels, t.Channels...)
+		if t.MoreChannels != nil {
+			channels = append(channels, t.MoreChannels...)
+		}
 		for _, c := range channels {
 			if c.Id == channelId {
 				return t.Id
@@ -549,12 +719,11 @@ func (m *MMClient) GetTeamFromChannel(channelId string) string {
 func (m *MMClient) GetLastViewedAt(channelId string) int64 {
 	m.RLock()
 	defer m.RUnlock()
-	res, err := m.Client.GetChannel(channelId, "")
-	if err != nil {
+	res, resp := m.Client.GetChannelMember(channelId, m.User.Id, "")
+	if resp.Error != nil {
 		return model.GetMillis()
 	}
-	data := res.Data.(*model.ChannelData)
-	return data.Member.LastViewedAt
+	return res.LastViewedAt
 }
 
 func (m *MMClient) GetUsers() map[string]*model.User {
@@ -568,42 +737,66 @@ func (m *MMClient) GetUsers() map[string]*model.User {
 }
 
 func (m *MMClient) GetUser(userId string) *model.User {
-	m.RLock()
-	defer m.RUnlock()
+	m.Lock()
+	defer m.Unlock()
+	_, ok := m.Users[userId]
+	if !ok {
+		res, resp := m.Client.GetUser(userId, "")
+		if resp.Error != nil {
+			return nil
+		}
+		m.Users[userId] = res
+	}
 	return m.Users[userId]
 }
 
+func (m *MMClient) GetUserName(userId string) string {
+	user := m.GetUser(userId)
+	if user != nil {
+		return user.Username
+	}
+	return ""
+}
+
 func (m *MMClient) GetStatus(userId string) string {
-	res, err := m.Client.GetStatuses()
-	if err != nil {
+	res, resp := m.Client.GetUserStatus(userId, "")
+	if resp.Error != nil {
 		return ""
 	}
-	status := res.Data.(map[string]string)
-	if status[userId] == model.STATUS_AWAY {
+	if res.Status == model.STATUS_AWAY {
 		return "away"
 	}
-	if status[userId] == model.STATUS_ONLINE {
+	if res.Status == model.STATUS_ONLINE {
 		return "online"
 	}
 	return "offline"
 }
 
+func (m *MMClient) UpdateStatus(userId string, status string) error {
+	_, resp := m.Client.UpdateUserStatus(userId, &model.Status{Status: status})
+	if resp.Error != nil {
+		return resp.Error
+	}
+	return nil
+}
+
 func (m *MMClient) GetStatuses() map[string]string {
-	var ok bool
+	var ids []string
 	statuses := make(map[string]string)
-	res, err := m.Client.GetStatuses()
-	if err != nil {
+	for id := range m.Users {
+		ids = append(ids, id)
+	}
+	res, resp := m.Client.GetUsersStatusesByIds(ids)
+	if resp.Error != nil {
 		return statuses
 	}
-	if statuses, ok = res.Data.(map[string]string); ok {
-		for userId, status := range statuses {
-			statuses[userId] = "offline"
-			if status == model.STATUS_AWAY {
-				statuses[userId] = "away"
-			}
-			if status == model.STATUS_ONLINE {
-				statuses[userId] = "online"
-			}
+	for _, status := range res {
+		statuses[status.UserId] = "offline"
+		if status.Status == model.STATUS_AWAY {
+			statuses[status.UserId] = "away"
+		}
+		if status.Status == model.STATUS_ONLINE {
+			statuses[status.UserId] = "online"
 		}
 	}
 	return statuses
@@ -613,7 +806,21 @@ func (m *MMClient) GetTeamId() string {
 	return m.Team.Id
 }
 
+func (m *MMClient) UploadFile(data []byte, channelId string, filename string) (string, error) {
+	f, resp := m.Client.UploadFile(data, channelId, filename)
+	if resp.Error != nil {
+		return "", resp.Error
+	}
+	return f.FileInfos[0].Id, nil
+}
+
 func (m *MMClient) StatusLoop() {
+	retries := 0
+	backoff := time.Second * 60
+	if m.OnWsConnect != nil {
+		m.OnWsConnect()
+	}
+	m.log.Debug("StatusLoop:", m.OnWsConnect)
 	for {
 		if m.WsQuit {
 			return
@@ -624,13 +831,28 @@ func (m *MMClient) StatusLoop() {
 			select {
 			case <-m.WsPingChan:
 				m.log.Debug("WS PONG received")
+				backoff = time.Second * 60
 			case <-time.After(time.Second * 5):
-				m.Logout()
-				m.WsQuit = false
-				m.Login()
+				if retries > 3 {
+					m.log.Debug("StatusLoop() timeout")
+					m.Logout()
+					m.WsQuit = false
+					err := m.Login()
+					if err != nil {
+						log.Errorf("Login failed: %#v", err)
+						break
+					}
+					if m.OnWsConnect != nil {
+						m.OnWsConnect()
+					}
+					go m.WsReceiver()
+				} else {
+					retries++
+					backoff = time.Second * 5
+				}
 			}
 		}
-		time.Sleep(time.Second * 60)
+		time.Sleep(backoff)
 	}
 }
 
@@ -638,36 +860,39 @@ func (m *MMClient) StatusLoop() {
 func (m *MMClient) initUser() error {
 	m.Lock()
 	defer m.Unlock()
-	initLoad, err := m.Client.GetInitialLoad()
-	if err != nil {
-		return err
-	}
-	initData := initLoad.Data.(*model.InitialLoad)
-	m.User = initData.User
 	// we only load all team data on initial login.
 	// all other updates are for channels from our (primary) team only.
 	//m.log.Debug("initUser(): loading all team data")
-	for _, v := range initData.Teams {
-		m.Client.SetTeamId(v.Id)
-		mmusers, err := m.Client.GetProfiles(0, 50000, "")
-		if err != nil {
-			return errors.New(err.DetailedError)
+	teams, resp := m.Client.GetTeamsForUser(m.User.Id, "")
+	if resp.Error != nil {
+		return resp.Error
+	}
+	for _, team := range teams {
+		mmusers, resp := m.Client.GetUsersInTeam(team.Id, 0, 50000, "")
+		if resp.Error != nil {
+			return errors.New(resp.Error.DetailedError)
 		}
-		t := &Team{Team: v, Users: mmusers.Data.(map[string]*model.User), Id: v.Id}
-		mmchannels, err := m.Client.GetChannels("")
-		if err != nil {
-			return errors.New(err.DetailedError)
+		usermap := make(map[string]*model.User)
+		for _, user := range mmusers {
+			usermap[user.Id] = user
 		}
-		t.Channels = mmchannels.Data.(*model.ChannelList)
-		mmchannels, err = m.Client.GetMoreChannels("")
-		if err != nil {
-			return errors.New(err.DetailedError)
+
+		t := &Team{Team: team, Users: usermap, Id: team.Id}
+
+		mmchannels, resp := m.Client.GetChannelsForTeamForUser(team.Id, m.User.Id, "")
+		if resp.Error != nil {
+			return resp.Error
 		}
-		t.MoreChannels = mmchannels.Data.(*model.ChannelList)
+		t.Channels = mmchannels
+		mmchannels, resp = m.Client.GetPublicChannelsForTeam(team.Id, 0, 5000, "")
+		if resp.Error != nil {
+			return resp.Error
+		}
+		t.MoreChannels = mmchannels
 		m.OtherTeams = append(m.OtherTeams, t)
-		if v.Name == m.Credentials.Team {
+		if team.Name == m.Credentials.Team {
 			m.Team = t
-			m.log.Debugf("initUser(): found our team %s (id: %s)", v.Name, v.Id)
+			m.log.Debugf("initUser(): found our team %s (id: %s)", team.Name, team.Id)
 		}
 		// add all users
 		for k, v := range t.Users {
@@ -687,3 +912,18 @@ func (m *MMClient) sendWSRequest(action string, data map[string]interface{}) err
 	m.WsClient.WriteJSON(req)
 	return nil
 }
+
+func supportedVersion(version string) bool {
+	if strings.HasPrefix(version, "3.8.0") ||
+		strings.HasPrefix(version, "3.9.0") ||
+		strings.HasPrefix(version, "3.10.0") ||
+		strings.HasPrefix(version, "4.") ||
+		strings.HasPrefix(version, "5.") {
+		return true
+	}
+	return false
+}
+
+func digestString(s string) string {
+	return fmt.Sprintf("%x", md5.Sum([]byte(s)))
+}

matterhook/matterhook.go

@@ -6,24 +6,27 @@ import (
 	"crypto/tls"
 	"encoding/json"
 	"fmt"
-	"github.com/gorilla/schema"
 	"io"
 	"io/ioutil"
 	"log"
 	"net"
 	"net/http"
 	"time"
+
+	"github.com/gorilla/schema"
+	"github.com/nlopes/slack"
 )
 
 // OMessage for mattermost incoming webhook. (send to mattermost)
 type OMessage struct {
-	Channel     string      `json:"channel,omitempty"`
-	IconURL     string      `json:"icon_url,omitempty"`
-	IconEmoji   string      `json:"icon_emoji,omitempty"`
-	UserName    string      `json:"username,omitempty"`
-	Text        string      `json:"text"`
-	Attachments interface{} `json:"attachments,omitempty"`
-	Type        string      `json:"type,omitempty"`
+	Channel     string                 `json:"channel,omitempty"`
+	IconURL     string                 `json:"icon_url,omitempty"`
+	IconEmoji   string                 `json:"icon_emoji,omitempty"`
+	UserName    string                 `json:"username,omitempty"`
+	Text        string                 `json:"text"`
+	Attachments []slack.Attachment     `json:"attachments,omitempty"`
+	Type        string                 `json:"type,omitempty"`
+	Props       map[string]interface{} `json:"props"`
 }
 
 // IMessage for mattermost outgoing webhook. (received from mattermost)
@@ -43,6 +46,7 @@ type IMessage struct {
 	ServiceId   string `schema:"service_id"`
 	Text        string `schema:"text"`
 	TriggerWord string `schema:"trigger_word"`
+	FileIDs     string `schema:"file_ids"`
 }
 
 // Client for Mattermost.
@@ -134,12 +138,11 @@ func (c *Client) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 
 // Receive returns an incoming message from mattermost outgoing webhooks URL.
 func (c *Client) Receive() IMessage {
-	for {
-		select {
-		case msg := <-c.In:
-			return msg
-		}
+	var msg IMessage
+	for msg := range c.In {
+		return msg
 	}
+	return msg
 }
 
 // Send sends a msg to mattermost incoming webhooks URL.

migration.md

@@ -1,50 +0,0 @@
-# Breaking changes from 0.4 to 0.5 for matterbridge (webhooks version)
-## IRC section
-### Server
-Port removed, added to server
-```
-server="irc.freenode.net"
-port=6667
-```
-changed to
-```
-server="irc.freenode.net:6667"
-```
-### Channel
-Removed see Channels section below
-
-### UseSlackCircumfix=true
-Removed, can be done by using ```RemoteNickFormat="<{NICK}> "```
-
-## Mattermost section
-### BindAddress
-Port removed, added to BindAddress
-
-```
-BindAddress="0.0.0.0"
-port=9999
-```
-
-changed to
-
-```
-BindAddress="0.0.0.0:9999"
-```
-
-### Token
-Removed
-
-## Channels section
-```
-[Token "outgoingwebhooktoken1"] 
-IRCChannel="#off-topic"
-MMChannel="off-topic"
-```
-
-changed to
-
-```
-[Channel "channelnameofchoice"] 
-IRC="#off-topic"
-Mattermost="off-topic"
-```

vendor/github.com/42wim/go-gitter/gitter.go

@@ -205,17 +205,43 @@ func (gitter *Gitter) GetMessage(roomID, messageID string) (*Message, error) {
 }
 
 // SendMessage sends a message to a room
-func (gitter *Gitter) SendMessage(roomID, text string) error {
+func (gitter *Gitter) SendMessage(roomID, text string) (*Message, error) {
 
 	message := Message{Text: text}
 	body, _ := json.Marshal(message)
-	_, err := gitter.post(gitter.config.apiBaseURL+"rooms/"+roomID+"/chatMessages", body)
+	response, err := gitter.post(gitter.config.apiBaseURL+"rooms/"+roomID+"/chatMessages", body)
 	if err != nil {
 		gitter.log(err)
-		return err
+		return nil, err
 	}
 
-	return nil
+	err = json.Unmarshal(response, &message)
+	if err != nil {
+		gitter.log(err)
+		return nil, err
+	}
+
+	return &message, nil
+}
+
+// UpdateMessage updates a message in a room
+func (gitter *Gitter) UpdateMessage(roomID, msgID, text string) (*Message, error) {
+
+	message := Message{Text: text}
+	body, _ := json.Marshal(message)
+	response, err := gitter.put(gitter.config.apiBaseURL+"rooms/"+roomID+"/chatMessages/"+msgID, body)
+	if err != nil {
+		gitter.log(err)
+		return nil, err
+	}
+
+	err = json.Unmarshal(response, &message)
+	if err != nil {
+		gitter.log(err)
+		return nil, err
+	}
+
+	return &message, nil
 }
 
 // JoinRoom joins a room
@@ -265,7 +291,7 @@ func (gitter *Gitter) SearchRooms(room string) ([]Room, error) {
 		Results []Room `json:"results"`
 	}
 
-	response, err := gitter.get(gitter.config.apiBaseURL + "rooms?q=" + room )
+	response, err := gitter.get(gitter.config.apiBaseURL + "rooms?q=" + room)
 
 	if err != nil {
 		gitter.log(err)
@@ -414,6 +440,39 @@ func (gitter *Gitter) post(url string, body []byte) ([]byte, error) {
 	return result, nil
 }
 
+func (gitter *Gitter) put(url string, body []byte) ([]byte, error) {
+	r, err := http.NewRequest("PUT", url, bytes.NewBuffer(body))
+	if err != nil {
+		gitter.log(err)
+		return nil, err
+	}
+
+	r.Header.Set("Content-Type", "application/json")
+	r.Header.Set("Accept", "application/json")
+	r.Header.Set("Authorization", "Bearer "+gitter.config.token)
+
+	resp, err := gitter.config.client.Do(r)
+	if err != nil {
+		gitter.log(err)
+		return nil, err
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode != http.StatusOK {
+		err = APIError{What: fmt.Sprintf("Status code: %v", resp.StatusCode)}
+		gitter.log(err)
+		return nil, err
+	}
+
+	result, err := ioutil.ReadAll(resp.Body)
+	if err != nil {
+		gitter.log(err)
+		return nil, err
+	}
+
+	return result, nil
+}
+
 func (gitter *Gitter) delete(url string) ([]byte, error) {
 	r, err := http.NewRequest("delete", url, nil)
 	if err != nil {

vendor/github.com/42wim/go-gitter/stream.go

@@ -47,13 +47,13 @@ Loop:
 			}
 			break Loop
 		}
-
+		
 		resp := stream.getResponse()
 		if resp.StatusCode != 200 {
 			gitter.log(fmt.Sprintf("Unexpected response code %v", resp.StatusCode))
 			continue
 		}
-
+		
 		//"The JSON stream returns messages as JSON objects that are delimited by carriage return (\r)" <- Not true crap it's (\n) only
 		reader = bufio.NewReader(resp.Body)
 		line, err := reader.ReadBytes('\n')
@@ -112,6 +112,7 @@ type Stream struct {
 
 func (stream *Stream) destroy() {
 	close(stream.Event)
+	stream.streamConnection.currentRetries = 0
 }
 
 type Event struct {
@@ -135,10 +136,11 @@ func (stream *Stream) connect() {
 	}
 
 	res, err := stream.gitter.getResponse(stream.url, stream)
-	if stream.streamConnection.canceled {
-		// do nothing
-	} else if err != nil || res.StatusCode != 200 {
-		stream.gitter.log("Failed to get response, trying reconnect ")
+	if err != nil || res.StatusCode != 200 {
+		stream.gitter.log("Failed to get response, trying reconnect")
+		if res != nil {
+			stream.gitter.log(fmt.Sprintf("Status code: %v", res.StatusCode))
+		}
 		stream.gitter.log(err)
 
 		// sleep and wait
@@ -161,9 +163,6 @@ type streamConnection struct {
 	// connection was closed
 	closed bool
 
-	// canceled
-	canceled bool
-
 	// wait time till next try
 	wait time.Duration
 
@@ -192,13 +191,10 @@ func (stream *Stream) Close() {
 		stream.gitter.log("Stream connection close request")
 		switch transport := stream.gitter.config.client.Transport.(type) {
 		case *httpclient.Transport:
-			stream.streamConnection.canceled = true
 			transport.CancelRequest(conn.request)
 		default:
 		}
-
 	}
-	conn.currentRetries = 0
 }
 
 func (stream *Stream) isClosed() bool {

vendor/github.com/42wim/go-ircevent/irc.go

@@ -87,6 +87,17 @@ func (irc *Connection) readLoop() {
 	}
 }
 
+// Unescape tag values as defined in the IRCv3.2 message tags spec
+// http://ircv3.net/specs/core/message-tags-3.2.html
+func unescapeTagValue(value string) string {
+	value = strings.Replace(value, "\\:", ";", -1)
+	value = strings.Replace(value, "\\s", " ", -1)
+	value = strings.Replace(value, "\\\\", "\\", -1)
+	value = strings.Replace(value, "\\r", "\r", -1)
+	value = strings.Replace(value, "\\n", "\n", -1)
+	return value
+}
+
 //Parse raw irc messages
 func parseToEvent(msg string) (*Event, error) {
 	msg = strings.TrimSuffix(msg, "\n") //Remove \r\n
@@ -95,6 +106,26 @@ func parseToEvent(msg string) (*Event, error) {
 	if len(msg) < 5 {
 		return nil, errors.New("Malformed msg from server")
 	}
+
+	if msg[0] == '@' {
+		// IRCv3 Message Tags
+		if i := strings.Index(msg, " "); i > -1 {
+			event.Tags = make(map[string]string)
+			tags := strings.Split(msg[1:i], ";")
+			for _, data := range tags {
+				parts := strings.SplitN(data, "=", 2)
+				if len(parts) == 1 {
+					event.Tags[parts[0]] = ""
+				} else {
+					event.Tags[parts[0]] = unescapeTagValue(parts[1])
+				}
+			}
+			msg = msg[i+1 : len(msg)]
+		} else {
+			return nil, errors.New("Malformed msg from server")
+		}
+	}
+
 	if msg[0] == ':' {
 		if i := strings.Index(msg, " "); i > -1 {
 			event.Source = msg[1:i]
@@ -196,12 +227,17 @@ func (irc *Connection) isQuitting() bool {
 // Main loop to control the connection.
 func (irc *Connection) Loop() {
 	errChan := irc.ErrorChan()
+	connTime := time.Now()
 	for !irc.isQuitting() {
 		err := <-errChan
 		close(irc.end)
 		irc.Wait()
 		for !irc.isQuitting() {
 			irc.Log.Printf("Error, disconnected: %s\n", err)
+			if time.Now().Sub(connTime) < time.Second*5 {
+				irc.Log.Println("Rreconnecting too fast, sleeping 60 seconds")
+				time.Sleep(60 * time.Second)
+			}
 			if err = irc.Reconnect(); err != nil {
 				irc.Log.Printf("Error while reconnecting: %s\n", err)
 				time.Sleep(60 * time.Second)
@@ -210,6 +246,7 @@ func (irc *Connection) Loop() {
 				break
 			}
 		}
+		connTime = time.Now()
 	}
 }
 
@@ -430,26 +467,84 @@ func (irc *Connection) Connect(server string) error {
 		irc.pwrite <- fmt.Sprintf("PASS %s\r\n", irc.Password)
 	}
 
-	resChan := make(chan *SASLResult)
+	err = irc.negotiateCaps()
+	if err != nil {
+		return err
+	}
+
+	irc.pwrite <- fmt.Sprintf("NICK %s\r\n", irc.nick)
+	irc.pwrite <- fmt.Sprintf("USER %s 0.0.0.0 0.0.0.0 :%s\r\n", irc.user, irc.user)
+	return nil
+}
+
+// Negotiate IRCv3 capabilities
+func (irc *Connection) negotiateCaps() error {
+	saslResChan := make(chan *SASLResult)
+	if irc.UseSASL {
+		irc.RequestCaps = append(irc.RequestCaps, "sasl")
+		irc.setupSASLCallbacks(saslResChan)
+	}
+
+	if len(irc.RequestCaps) == 0 {
+		return nil
+	}
+
+	cap_chan := make(chan bool, len(irc.RequestCaps))
+	irc.AddCallback("CAP", func(e *Event) {
+		if len(e.Arguments) != 3 {
+			return
+		}
+		command := e.Arguments[1]
+
+		if command == "LS" {
+			missing_caps := len(irc.RequestCaps)
+			for _, cap_name := range strings.Split(e.Arguments[2], " ") {
+				for _, req_cap := range irc.RequestCaps {
+					if cap_name == req_cap {
+						irc.pwrite <- fmt.Sprintf("CAP REQ :%s\r\n", cap_name)
+						missing_caps--
+					}
+				}
+			}
+
+			for i := 0; i < missing_caps; i++ {
+				cap_chan <- true
+			}
+		} else if command == "ACK" || command == "NAK" {
+			for _, cap_name := range strings.Split(strings.TrimSpace(e.Arguments[2]), " ") {
+				if cap_name == "" {
+					continue
+				}
+
+				if command == "ACK" {
+					irc.AcknowledgedCaps = append(irc.AcknowledgedCaps, cap_name)
+				}
+				cap_chan <- true
+			}
+		}
+	})
+
+	irc.pwrite <- "CAP LS\r\n"
+
 	if irc.UseSASL {
-		irc.setupSASLCallbacks(resChan)
-		irc.pwrite <- fmt.Sprintf("CAP LS\r\n")
-		// request SASL
-		irc.pwrite <- fmt.Sprintf("CAP REQ :sasl\r\n")
-		// if sasl request doesn't complete in 15 seconds, close chan and timeout
 		select {
-		case res := <-resChan:
+		case res := <-saslResChan:
 			if res.Failed {
-				close(resChan)
+				close(saslResChan)
 				return res.Err
 			}
 		case <-time.After(time.Second * 15):
-			close(resChan)
+			close(saslResChan)
 			return errors.New("SASL setup timed out. This shouldn't happen.")
 		}
 	}
-	irc.pwrite <- fmt.Sprintf("NICK %s\r\n", irc.nick)
-	irc.pwrite <- fmt.Sprintf("USER %s 0.0.0.0 0.0.0.0 :%s\r\n", irc.user, irc.user)
+
+	// Wait for all capabilities to be ACKed or NAKed before ending negotiation
+	for i := 0; i < len(irc.RequestCaps); i++ {
+		<-cap_chan
+	}
+	irc.pwrite <- fmt.Sprintf("CAP END\r\n")
+
 	return nil
 }
 

vendor/github.com/42wim/go-ircevent/irc_sasl.go

@@ -43,7 +43,6 @@ func (irc *Connection) setupSASLCallbacks(result chan<- *SASLResult) {
 		result <- &SASLResult{true, errors.New(e.Arguments[1])}
 	})
 	irc.AddCallback("903", func(e *Event) {
-		irc.SendRaw("CAP END")
 		result <- &SASLResult{false, nil}
 	})
 	irc.AddCallback("904", func(e *Event) {

vendor/github.com/42wim/go-ircevent/irc_struct.go

@@ -15,20 +15,22 @@ import (
 type Connection struct {
 	sync.Mutex
 	sync.WaitGroup
-	Debug        bool
-	Error        chan error
-	Password     string
-	UseTLS       bool
-	UseSASL      bool
-	SASLLogin    string
-	SASLPassword string
-	SASLMech     string
-	TLSConfig    *tls.Config
-	Version      string
-	Timeout      time.Duration
-	PingFreq     time.Duration
-	KeepAlive    time.Duration
-	Server       string
+	Debug            bool
+	Error            chan error
+	Password         string
+	UseTLS           bool
+	UseSASL          bool
+	RequestCaps      []string
+	AcknowledgedCaps []string
+	SASLLogin        string
+	SASLPassword     string
+	SASLMech         string
+	TLSConfig        *tls.Config
+	Version          string
+	Timeout          time.Duration
+	PingFreq         time.Duration
+	KeepAlive        time.Duration
+	Server           string
 
 	socket net.Conn
 	pwrite chan string
@@ -59,6 +61,7 @@ type Event struct {
 	Source     string //<host>
 	User       string //<usr>
 	Arguments  []string
+	Tags       map[string]string
 	Connection *Connection
 }
 

vendor/github.com/BurntSushi/toml/doc.go

@@ -4,7 +4,7 @@ files via reflection. There is also support for delaying decoding with
 the Primitive type, and querying the set of keys in a TOML document with the
 MetaData type.
 
-The specification implemented: https://github.com/mojombo/toml
+The specification implemented: https://github.com/toml-lang/toml
 
 The sub-command github.com/BurntSushi/toml/cmd/tomlv can be used to verify
 whether a file is a valid TOML document. It can also be used to print the

vendor/github.com/BurntSushi/toml/encode.go

@@ -241,7 +241,7 @@ func (enc *Encoder) eArrayOfTables(key Key, rv reflect.Value) {
 func (enc *Encoder) eTable(key Key, rv reflect.Value) {
 	panicIfInvalidKey(key)
 	if len(key) == 1 {
-		// Output an extra new line between top-level tables.
+		// Output an extra newline between top-level tables.
 		// (The newline isn't written if nothing else has been written though.)
 		enc.newline()
 	}

vendor/github.com/BurntSushi/toml/lex.go

@@ -30,24 +30,28 @@ const (
 	itemArrayTableEnd
 	itemKeyStart
 	itemCommentStart
+	itemInlineTableStart
+	itemInlineTableEnd
 )
 
 const (
-	eof             = 0
-	tableStart      = '['
-	tableEnd        = ']'
-	arrayTableStart = '['
-	arrayTableEnd   = ']'
-	tableSep        = '.'
-	keySep          = '='
-	arrayStart      = '['
-	arrayEnd        = ']'
-	arrayValTerm    = ','
-	commentStart    = '#'
-	stringStart     = '"'
-	stringEnd       = '"'
-	rawStringStart  = '\''
-	rawStringEnd    = '\''
+	eof              = 0
+	comma            = ','
+	tableStart       = '['
+	tableEnd         = ']'
+	arrayTableStart  = '['
+	arrayTableEnd    = ']'
+	tableSep         = '.'
+	keySep           = '='
+	arrayStart       = '['
+	arrayEnd         = ']'
+	commentStart     = '#'
+	stringStart      = '"'
+	stringEnd        = '"'
+	rawStringStart   = '\''
+	rawStringEnd     = '\''
+	inlineTableStart = '{'
+	inlineTableEnd   = '}'
 )
 
 type stateFn func(lx *lexer) stateFn
@@ -56,11 +60,18 @@ type lexer struct {
 	input string
 	start int
 	pos   int
-	width int
 	line  int
 	state stateFn
 	items chan item
 
+	// Allow for backing up up to three runes.
+	// This is necessary because TOML contains 3-rune tokens (""" and ''').
+	prevWidths [3]int
+	nprev      int // how many of prevWidths are in use
+	// If we emit an eof, we can still back up, but it is not OK to call
+	// next again.
+	atEOF bool
+
 	// A stack of state functions used to maintain context.
 	// The idea is to reuse parts of the state machine in various places.
 	// For example, values can appear at the top level or within arbitrarily
@@ -88,7 +99,7 @@ func (lx *lexer) nextItem() item {
 
 func lex(input string) *lexer {
 	lx := &lexer{
-		input: input + "\n",
+		input: input,
 		state: lexTop,
 		line:  1,
 		items: make(chan item, 10),
@@ -103,7 +114,7 @@ func (lx *lexer) push(state stateFn) {
 
 func (lx *lexer) pop() stateFn {
 	if len(lx.stack) == 0 {
-		return lx.errorf("BUG in lexer: no states to pop.")
+		return lx.errorf("BUG in lexer: no states to pop")
 	}
 	last := lx.stack[len(lx.stack)-1]
 	lx.stack = lx.stack[0 : len(lx.stack)-1]
@@ -125,16 +136,25 @@ func (lx *lexer) emitTrim(typ itemType) {
 }
 
 func (lx *lexer) next() (r rune) {
+	if lx.atEOF {
+		panic("next called after EOF")
+	}
 	if lx.pos >= len(lx.input) {
-		lx.width = 0
+		lx.atEOF = true
 		return eof
 	}
 
 	if lx.input[lx.pos] == '\n' {
 		lx.line++
 	}
-	r, lx.width = utf8.DecodeRuneInString(lx.input[lx.pos:])
-	lx.pos += lx.width
+	lx.prevWidths[2] = lx.prevWidths[1]
+	lx.prevWidths[1] = lx.prevWidths[0]
+	if lx.nprev < 3 {
+		lx.nprev++
+	}
+	r, w := utf8.DecodeRuneInString(lx.input[lx.pos:])
+	lx.prevWidths[0] = w
+	lx.pos += w
 	return r
 }
 
@@ -143,9 +163,20 @@ func (lx *lexer) ignore() {
 	lx.start = lx.pos
 }
 
-// backup steps back one rune. Can be called only once per call of next.
+// backup steps back one rune. Can be called only twice between calls to next.
 func (lx *lexer) backup() {
-	lx.pos -= lx.width
+	if lx.atEOF {
+		lx.atEOF = false
+		return
+	}
+	if lx.nprev < 1 {
+		panic("backed up too far")
+	}
+	w := lx.prevWidths[0]
+	lx.prevWidths[0] = lx.prevWidths[1]
+	lx.prevWidths[1] = lx.prevWidths[2]
+	lx.nprev--
+	lx.pos -= w
 	if lx.pos < len(lx.input) && lx.input[lx.pos] == '\n' {
 		lx.line--
 	}
@@ -182,7 +213,7 @@ func (lx *lexer) skip(pred func(rune) bool) {
 
 // errorf stops all lexing by emitting an error and returning `nil`.
 // Note that any value that is a character is escaped if it's a special
-// character (new lines, tabs, etc.).
+// character (newlines, tabs, etc.).
 func (lx *lexer) errorf(format string, values ...interface{}) stateFn {
 	lx.items <- item{
 		itemError,
@@ -198,7 +229,6 @@ func lexTop(lx *lexer) stateFn {
 	if isWhitespace(r) || isNL(r) {
 		return lexSkip(lx, lexTop)
 	}
-
 	switch r {
 	case commentStart:
 		lx.push(lexTop)
@@ -207,7 +237,7 @@ func lexTop(lx *lexer) stateFn {
 		return lexTableStart
 	case eof:
 		if lx.pos > lx.start {
-			return lx.errorf("Unexpected EOF.")
+			return lx.errorf("unexpected EOF")
 		}
 		lx.emit(itemEOF)
 		return nil
@@ -222,12 +252,12 @@ func lexTop(lx *lexer) stateFn {
 
 // lexTopEnd is entered whenever a top-level item has been consumed. (A value
 // or a table.) It must see only whitespace, and will turn back to lexTop
-// upon a new line. If it sees EOF, it will quit the lexer successfully.
+// upon a newline. If it sees EOF, it will quit the lexer successfully.
 func lexTopEnd(lx *lexer) stateFn {
 	r := lx.next()
 	switch {
 	case r == commentStart:
-		// a comment will read to a new line for us.
+		// a comment will read to a newline for us.
 		lx.push(lexTop)
 		return lexCommentStart
 	case isWhitespace(r):
@@ -236,11 +266,11 @@ func lexTopEnd(lx *lexer) stateFn {
 		lx.ignore()
 		return lexTop
 	case r == eof:
-		lx.ignore()
-		return lexTop
+		lx.emit(itemEOF)
+		return nil
 	}
-	return lx.errorf("Expected a top-level item to end with a new line, "+
-		"comment or EOF, but got %q instead.", r)
+	return lx.errorf("expected a top-level item to end with a newline, "+
+		"comment, or EOF, but got %q instead", r)
 }
 
 // lexTable lexes the beginning of a table. Namely, it makes sure that
@@ -267,8 +297,8 @@ func lexTableEnd(lx *lexer) stateFn {
 
 func lexArrayTableEnd(lx *lexer) stateFn {
 	if r := lx.next(); r != arrayTableEnd {
-		return lx.errorf("Expected end of table array name delimiter %q, "+
-			"but got %q instead.", arrayTableEnd, r)
+		return lx.errorf("expected end of table array name delimiter %q, "+
+			"but got %q instead", arrayTableEnd, r)
 	}
 	lx.emit(itemArrayTableEnd)
 	return lexTopEnd
@@ -278,11 +308,11 @@ func lexTableNameStart(lx *lexer) stateFn {
 	lx.skip(isWhitespace)
 	switch r := lx.peek(); {
 	case r == tableEnd || r == eof:
-		return lx.errorf("Unexpected end of table name. (Table names cannot " +
-			"be empty.)")
+		return lx.errorf("unexpected end of table name " +
+			"(table names cannot be empty)")
 	case r == tableSep:
-		return lx.errorf("Unexpected table separator. (Table names cannot " +
-			"be empty.)")
+		return lx.errorf("unexpected table separator " +
+			"(table names cannot be empty)")
 	case r == stringStart || r == rawStringStart:
 		lx.ignore()
 		lx.push(lexTableNameEnd)
@@ -317,8 +347,8 @@ func lexTableNameEnd(lx *lexer) stateFn {
 	case r == tableEnd:
 		return lx.pop()
 	default:
-		return lx.errorf("Expected '.' or ']' to end table name, but got %q "+
-			"instead.", r)
+		return lx.errorf("expected '.' or ']' to end table name, "+
+			"but got %q instead", r)
 	}
 }
 
@@ -328,7 +358,7 @@ func lexKeyStart(lx *lexer) stateFn {
 	r := lx.peek()
 	switch {
 	case r == keySep:
-		return lx.errorf("Unexpected key separator %q.", keySep)
+		return lx.errorf("unexpected key separator %q", keySep)
 	case isWhitespace(r) || isNL(r):
 		lx.next()
 		return lexSkip(lx, lexKeyStart)
@@ -359,7 +389,7 @@ func lexBareKey(lx *lexer) stateFn {
 		lx.emit(itemText)
 		return lexKeyEnd
 	default:
-		return lx.errorf("Bare keys cannot contain %q.", r)
+		return lx.errorf("bare keys cannot contain %q", r)
 	}
 }
 
@@ -372,7 +402,7 @@ func lexKeyEnd(lx *lexer) stateFn {
 	case isWhitespace(r):
 		return lexSkip(lx, lexKeyEnd)
 	default:
-		return lx.errorf("Expected key separator %q, but got %q instead.",
+		return lx.errorf("expected key separator %q, but got %q instead",
 			keySep, r)
 	}
 }
@@ -381,9 +411,8 @@ func lexKeyEnd(lx *lexer) stateFn {
 // lexValue will ignore whitespace.
 // After a value is lexed, the last state on the next is popped and returned.
 func lexValue(lx *lexer) stateFn {
-	// We allow whitespace to precede a value, but NOT new lines.
-	// In array syntax, the array states are responsible for ignoring new
-	// lines.
+	// We allow whitespace to precede a value, but NOT newlines.
+	// In array syntax, the array states are responsible for ignoring newlines.
 	r := lx.next()
 	switch {
 	case isWhitespace(r):
@@ -397,6 +426,10 @@ func lexValue(lx *lexer) stateFn {
 		lx.ignore()
 		lx.emit(itemArray)
 		return lexArrayValue
+	case inlineTableStart:
+		lx.ignore()
+		lx.emit(itemInlineTableStart)
+		return lexInlineTableValue
 	case stringStart:
 		if lx.accept(stringStart) {
 			if lx.accept(stringStart) {
@@ -420,7 +453,7 @@ func lexValue(lx *lexer) stateFn {
 	case '+', '-':
 		return lexNumberStart
 	case '.': // special error case, be kind to users
-		return lx.errorf("Floats must start with a digit, not '.'.")
+		return lx.errorf("floats must start with a digit, not '.'")
 	}
 	if unicode.IsLetter(r) {
 		// Be permissive here; lexBool will give a nice error if the
@@ -430,11 +463,11 @@ func lexValue(lx *lexer) stateFn {
 		lx.backup()
 		return lexBool
 	}
-	return lx.errorf("Expected value but found %q instead.", r)
+	return lx.errorf("expected value but found %q instead", r)
 }
 
 // lexArrayValue consumes one value in an array. It assumes that '[' or ','
-// have already been consumed. All whitespace and new lines are ignored.
+// have already been consumed. All whitespace and newlines are ignored.
 func lexArrayValue(lx *lexer) stateFn {
 	r := lx.next()
 	switch {
@@ -443,10 +476,11 @@ func lexArrayValue(lx *lexer) stateFn {
 	case r == commentStart:
 		lx.push(lexArrayValue)
 		return lexCommentStart
-	case r == arrayValTerm:
-		return lx.errorf("Unexpected array value terminator %q.",
-			arrayValTerm)
+	case r == comma:
+		return lx.errorf("unexpected comma")
 	case r == arrayEnd:
+		// NOTE(caleb): The spec isn't clear about whether you can have
+		// a trailing comma or not, so we'll allow it.
 		return lexArrayEnd
 	}
 
@@ -455,8 +489,9 @@ func lexArrayValue(lx *lexer) stateFn {
 	return lexValue
 }
 
-// lexArrayValueEnd consumes the cruft between values of an array. Namely,
-// it ignores whitespace and expects either a ',' or a ']'.
+// lexArrayValueEnd consumes everything between the end of an array value and
+// the next value (or the end of the array): it ignores whitespace and newlines
+// and expects either a ',' or a ']'.
 func lexArrayValueEnd(lx *lexer) stateFn {
 	r := lx.next()
 	switch {
@@ -465,31 +500,88 @@ func lexArrayValueEnd(lx *lexer) stateFn {
 	case r == commentStart:
 		lx.push(lexArrayValueEnd)
 		return lexCommentStart
-	case r == arrayValTerm:
+	case r == comma:
 		lx.ignore()
 		return lexArrayValue // move on to the next value
 	case r == arrayEnd:
 		return lexArrayEnd
 	}
-	return lx.errorf("Expected an array value terminator %q or an array "+
-		"terminator %q, but got %q instead.", arrayValTerm, arrayEnd, r)
+	return lx.errorf(
+		"expected a comma or array terminator %q, but got %q instead",
+		arrayEnd, r,
+	)
 }
 
-// lexArrayEnd finishes the lexing of an array. It assumes that a ']' has
-// just been consumed.
+// lexArrayEnd finishes the lexing of an array.
+// It assumes that a ']' has just been consumed.
 func lexArrayEnd(lx *lexer) stateFn {
 	lx.ignore()
 	lx.emit(itemArrayEnd)
 	return lx.pop()
 }
 
+// lexInlineTableValue consumes one key/value pair in an inline table.
+// It assumes that '{' or ',' have already been consumed. Whitespace is ignored.
+func lexInlineTableValue(lx *lexer) stateFn {
+	r := lx.next()
+	switch {
+	case isWhitespace(r):
+		return lexSkip(lx, lexInlineTableValue)
+	case isNL(r):
+		return lx.errorf("newlines not allowed within inline tables")
+	case r == commentStart:
+		lx.push(lexInlineTableValue)
+		return lexCommentStart
+	case r == comma:
+		return lx.errorf("unexpected comma")
+	case r == inlineTableEnd:
+		return lexInlineTableEnd
+	}
+	lx.backup()
+	lx.push(lexInlineTableValueEnd)
+	return lexKeyStart
+}
+
+// lexInlineTableValueEnd consumes everything between the end of an inline table
+// key/value pair and the next pair (or the end of the table):
+// it ignores whitespace and expects either a ',' or a '}'.
+func lexInlineTableValueEnd(lx *lexer) stateFn {
+	r := lx.next()
+	switch {
+	case isWhitespace(r):
+		return lexSkip(lx, lexInlineTableValueEnd)
+	case isNL(r):
+		return lx.errorf("newlines not allowed within inline tables")
+	case r == commentStart:
+		lx.push(lexInlineTableValueEnd)
+		return lexCommentStart
+	case r == comma:
+		lx.ignore()
+		return lexInlineTableValue
+	case r == inlineTableEnd:
+		return lexInlineTableEnd
+	}
+	return lx.errorf("expected a comma or an inline table terminator %q, "+
+		"but got %q instead", inlineTableEnd, r)
+}
+
+// lexInlineTableEnd finishes the lexing of an inline table.
+// It assumes that a '}' has just been consumed.
+func lexInlineTableEnd(lx *lexer) stateFn {
+	lx.ignore()
+	lx.emit(itemInlineTableEnd)
+	return lx.pop()
+}
+
 // lexString consumes the inner contents of a string. It assumes that the
 // beginning '"' has already been consumed and ignored.
 func lexString(lx *lexer) stateFn {
 	r := lx.next()
 	switch {
+	case r == eof:
+		return lx.errorf("unexpected EOF")
 	case isNL(r):
-		return lx.errorf("Strings cannot contain new lines.")
+		return lx.errorf("strings cannot contain newlines")
 	case r == '\\':
 		lx.push(lexString)
 		return lexStringEscape
@@ -506,11 +598,12 @@ func lexString(lx *lexer) stateFn {
 // lexMultilineString consumes the inner contents of a string. It assumes that
 // the beginning '"""' has already been consumed and ignored.
 func lexMultilineString(lx *lexer) stateFn {
-	r := lx.next()
-	switch {
-	case r == '\\':
+	switch lx.next() {
+	case eof:
+		return lx.errorf("unexpected EOF")
+	case '\\':
 		return lexMultilineStringEscape
-	case r == stringEnd:
+	case stringEnd:
 		if lx.accept(stringEnd) {
 			if lx.accept(stringEnd) {
 				lx.backup()
@@ -534,8 +627,10 @@ func lexMultilineString(lx *lexer) stateFn {
 func lexRawString(lx *lexer) stateFn {
 	r := lx.next()
 	switch {
+	case r == eof:
+		return lx.errorf("unexpected EOF")
 	case isNL(r):
-		return lx.errorf("Strings cannot contain new lines.")
+		return lx.errorf("strings cannot contain newlines")
 	case r == rawStringEnd:
 		lx.backup()
 		lx.emit(itemRawString)
@@ -547,12 +642,13 @@ func lexRawString(lx *lexer) stateFn {
 }
 
 // lexMultilineRawString consumes a raw string. Nothing can be escaped in such
-// a string. It assumes that the beginning "'" has already been consumed and
+// a string. It assumes that the beginning "'''" has already been consumed and
 // ignored.
 func lexMultilineRawString(lx *lexer) stateFn {
-	r := lx.next()
-	switch {
-	case r == rawStringEnd:
+	switch lx.next() {
+	case eof:
+		return lx.errorf("unexpected EOF")
+	case rawStringEnd:
 		if lx.accept(rawStringEnd) {
 			if lx.accept(rawStringEnd) {
 				lx.backup()
@@ -605,10 +701,9 @@ func lexStringEscape(lx *lexer) stateFn {
 	case 'U':
 		return lexLongUnicodeEscape
 	}
-	return lx.errorf("Invalid escape character %q. Only the following "+
+	return lx.errorf("invalid escape character %q; only the following "+
 		"escape characters are allowed: "+
-		"\\b, \\t, \\n, \\f, \\r, \\\", \\/, \\\\, "+
-		"\\uXXXX and \\UXXXXXXXX.", r)
+		`\b, \t, \n, \f, \r, \", \\, \uXXXX, and \UXXXXXXXX`, r)
 }
 
 func lexShortUnicodeEscape(lx *lexer) stateFn {
@@ -616,8 +711,8 @@ func lexShortUnicodeEscape(lx *lexer) stateFn {
 	for i := 0; i < 4; i++ {
 		r = lx.next()
 		if !isHexadecimal(r) {
-			return lx.errorf("Expected four hexadecimal digits after '\\u', "+
-				"but got '%s' instead.", lx.current())
+			return lx.errorf(`expected four hexadecimal digits after '\u', `+
+				"but got %q instead", lx.current())
 		}
 	}
 	return lx.pop()
@@ -628,8 +723,8 @@ func lexLongUnicodeEscape(lx *lexer) stateFn {
 	for i := 0; i < 8; i++ {
 		r = lx.next()
 		if !isHexadecimal(r) {
-			return lx.errorf("Expected eight hexadecimal digits after '\\U', "+
-				"but got '%s' instead.", lx.current())
+			return lx.errorf(`expected eight hexadecimal digits after '\U', `+
+				"but got %q instead", lx.current())
 		}
 	}
 	return lx.pop()
@@ -647,9 +742,9 @@ func lexNumberOrDateStart(lx *lexer) stateFn {
 	case 'e', 'E':
 		return lexFloat
 	case '.':
-		return lx.errorf("Floats must start with a digit, not '.'.")
+		return lx.errorf("floats must start with a digit, not '.'")
 	}
-	return lx.errorf("Expected a digit but got %q.", r)
+	return lx.errorf("expected a digit but got %q", r)
 }
 
 // lexNumberOrDate consumes either an integer, float or datetime.
@@ -697,9 +792,9 @@ func lexNumberStart(lx *lexer) stateFn {
 	r := lx.next()
 	if !isDigit(r) {
 		if r == '.' {
-			return lx.errorf("Floats must start with a digit, not '.'.")
+			return lx.errorf("floats must start with a digit, not '.'")
 		}
-		return lx.errorf("Expected a digit but got %q.", r)
+		return lx.errorf("expected a digit but got %q", r)
 	}
 	return lexNumber
 }
@@ -757,7 +852,7 @@ func lexBool(lx *lexer) stateFn {
 		lx.emit(itemBool)
 		return lx.pop()
 	}
-	return lx.errorf("Expected value but found %q instead.", s)
+	return lx.errorf("expected value but found %q instead", s)
 }
 
 // lexCommentStart begins the lexing of a comment. It will emit
@@ -769,7 +864,7 @@ func lexCommentStart(lx *lexer) stateFn {
 }
 
 // lexComment lexes an entire comment. It assumes that '#' has been consumed.
-// It will consume *up to* the first new line character, and pass control
+// It will consume *up to* the first newline character, and pass control
 // back to the last state on the stack.
 func lexComment(lx *lexer) stateFn {
 	r := lx.peek()

vendor/github.com/BurntSushi/toml/parse.go

@@ -269,6 +269,41 @@ func (p *parser) value(it item) (interface{}, tomlType) {
 			types = append(types, typ)
 		}
 		return array, p.typeOfArray(types)
+	case itemInlineTableStart:
+		var (
+			hash         = make(map[string]interface{})
+			outerContext = p.context
+			outerKey     = p.currentKey
+		)
+
+		p.context = append(p.context, p.currentKey)
+		p.currentKey = ""
+		for it := p.next(); it.typ != itemInlineTableEnd; it = p.next() {
+			if it.typ != itemKeyStart {
+				p.bug("Expected key start but instead found %q, around line %d",
+					it.val, p.approxLine)
+			}
+			if it.typ == itemCommentStart {
+				p.expect(itemText)
+				continue
+			}
+
+			// retrieve key
+			k := p.next()
+			p.approxLine = k.line
+			kname := p.keyString(k)
+
+			// retrieve value
+			p.currentKey = kname
+			val, typ := p.value(p.next())
+			// make sure we keep metadata up to date
+			p.setType(kname, typ)
+			p.ordered = append(p.ordered, p.context.add(p.currentKey))
+			hash[kname] = val
+		}
+		p.context = outerContext
+		p.currentKey = outerKey
+		return hash, tomlHash
 	}
 	p.bug("Unexpected value type: %s", it.typ)
 	panic("unreachable")

vendor/github.com/Philipp15b/go-steam/LICENSE.txt

@@ -0,0 +1,26 @@
+Copyright (c) 2014 The go-steam Authors. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+   * Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+   * Redistributions in binary form must reproduce the above
+copyright notice, this list of conditions and the following disclaimer
+in the documentation and/or other materials provided with the
+distribution.
+   * The names of its contributors may not be used to endorse or promote
+products derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

vendor/github.com/Philipp15b/go-steam/auth.go

@@ -0,0 +1,178 @@
+package steam
+
+import (
+	"crypto/sha1"
+	. "github.com/Philipp15b/go-steam/protocol"
+	. "github.com/Philipp15b/go-steam/protocol/protobuf"
+	. "github.com/Philipp15b/go-steam/protocol/steamlang"
+	. "github.com/Philipp15b/go-steam/steamid"
+	"github.com/golang/protobuf/proto"
+	"sync/atomic"
+	"time"
+)
+
+type Auth struct {
+	client  *Client
+	details *LogOnDetails
+}
+
+type SentryHash []byte
+
+type LogOnDetails struct {
+	Username       string
+	Password       string
+	AuthCode       string
+	TwoFactorCode  string
+	SentryFileHash SentryHash
+}
+
+// Log on with the given details. You must always specify username and
+// password. For the first login, don't set an authcode or a hash and you'll receive an error
+// and Steam will send you an authcode. Then you have to login again, this time with the authcode.
+// Shortly after logging in, you'll receive a MachineAuthUpdateEvent with a hash which allows
+// you to login without using an authcode in the future.
+//
+// If you don't use Steam Guard, username and password are enough.
+func (a *Auth) LogOn(details *LogOnDetails) {
+	if len(details.Username) == 0 || len(details.Password) == 0 {
+		panic("Username and password must be set!")
+	}
+
+	logon := new(CMsgClientLogon)
+	logon.AccountName = &details.Username
+	logon.Password = &details.Password
+	if details.AuthCode != "" {
+		logon.AuthCode = proto.String(details.AuthCode)
+	}
+	if details.TwoFactorCode != "" {
+		logon.TwoFactorCode = proto.String(details.TwoFactorCode)
+	}
+	logon.ClientLanguage = proto.String("english")
+	logon.ProtocolVersion = proto.Uint32(MsgClientLogon_CurrentProtocol)
+	logon.ShaSentryfile = details.SentryFileHash
+
+	atomic.StoreUint64(&a.client.steamId, uint64(NewIdAdv(0, 1, int32(EUniverse_Public), int32(EAccountType_Individual))))
+
+	a.client.Write(NewClientMsgProtobuf(EMsg_ClientLogon, logon))
+}
+
+func (a *Auth) HandlePacket(packet *Packet) {
+	switch packet.EMsg {
+	case EMsg_ClientLogOnResponse:
+		a.handleLogOnResponse(packet)
+	case EMsg_ClientNewLoginKey:
+		a.handleLoginKey(packet)
+	case EMsg_ClientSessionToken:
+	case EMsg_ClientLoggedOff:
+		a.handleLoggedOff(packet)
+	case EMsg_ClientUpdateMachineAuth:
+		a.handleUpdateMachineAuth(packet)
+	case EMsg_ClientAccountInfo:
+		a.handleAccountInfo(packet)
+	case EMsg_ClientWalletInfoUpdate:
+	case EMsg_ClientRequestWebAPIAuthenticateUserNonceResponse:
+	case EMsg_ClientMarketingMessageUpdate:
+	}
+}
+
+func (a *Auth) handleLogOnResponse(packet *Packet) {
+	if !packet.IsProto {
+		a.client.Fatalf("Got non-proto logon response!")
+		return
+	}
+
+	body := new(CMsgClientLogonResponse)
+	msg := packet.ReadProtoMsg(body)
+
+	result := EResult(body.GetEresult())
+	if result == EResult_OK {
+		atomic.StoreInt32(&a.client.sessionId, msg.Header.Proto.GetClientSessionid())
+		atomic.StoreUint64(&a.client.steamId, msg.Header.Proto.GetSteamid())
+		a.client.Web.webLoginKey = *body.WebapiAuthenticateUserNonce
+
+		go a.client.heartbeatLoop(time.Duration(body.GetOutOfGameHeartbeatSeconds()))
+
+		a.client.Emit(&LoggedOnEvent{
+			Result:                    EResult(body.GetEresult()),
+			ExtendedResult:            EResult(body.GetEresultExtended()),
+			OutOfGameSecsPerHeartbeat: body.GetOutOfGameHeartbeatSeconds(),
+			InGameSecsPerHeartbeat:    body.GetInGameHeartbeatSeconds(),
+			PublicIp:                  body.GetPublicIp(),
+			ServerTime:                body.GetRtime32ServerTime(),
+			AccountFlags:              EAccountFlags(body.GetAccountFlags()),
+			ClientSteamId:             SteamId(body.GetClientSuppliedSteamid()),
+			EmailDomain:               body.GetEmailDomain(),
+			CellId:                    body.GetCellId(),
+			CellIdPingThreshold:       body.GetCellIdPingThreshold(),
+			Steam2Ticket:              body.GetSteam2Ticket(),
+			UsePics:                   body.GetUsePics(),
+			WebApiUserNonce:           body.GetWebapiAuthenticateUserNonce(),
+			IpCountryCode:             body.GetIpCountryCode(),
+			VanityUrl:                 body.GetVanityUrl(),
+			NumLoginFailuresToMigrate: body.GetCountLoginfailuresToMigrate(),
+			NumDisconnectsToMigrate:   body.GetCountDisconnectsToMigrate(),
+		})
+	} else if result == EResult_Fail || result == EResult_ServiceUnavailable || result == EResult_TryAnotherCM {
+		// some error on Steam's side, we'll get an EOF later
+	} else {
+		a.client.Emit(&LogOnFailedEvent{
+			Result: EResult(body.GetEresult()),
+		})
+		a.client.Disconnect()
+	}
+}
+
+func (a *Auth) handleLoginKey(packet *Packet) {
+	body := new(CMsgClientNewLoginKey)
+	packet.ReadProtoMsg(body)
+	a.client.Write(NewClientMsgProtobuf(EMsg_ClientNewLoginKeyAccepted, &CMsgClientNewLoginKeyAccepted{
+		UniqueId: proto.Uint32(body.GetUniqueId()),
+	}))
+	a.client.Emit(&LoginKeyEvent{
+		UniqueId: body.GetUniqueId(),
+		LoginKey: body.GetLoginKey(),
+	})
+}
+
+func (a *Auth) handleLoggedOff(packet *Packet) {
+	result := EResult_Invalid
+	if packet.IsProto {
+		body := new(CMsgClientLoggedOff)
+		packet.ReadProtoMsg(body)
+		result = EResult(body.GetEresult())
+	} else {
+		body := new(MsgClientLoggedOff)
+		packet.ReadClientMsg(body)
+		result = body.Result
+	}
+	a.client.Emit(&LoggedOffEvent{Result: result})
+}
+
+func (a *Auth) handleUpdateMachineAuth(packet *Packet) {
+	body := new(CMsgClientUpdateMachineAuth)
+	packet.ReadProtoMsg(body)
+	hash := sha1.New()
+	hash.Write(packet.Data)
+	sha := hash.Sum(nil)
+
+	msg := NewClientMsgProtobuf(EMsg_ClientUpdateMachineAuthResponse, &CMsgClientUpdateMachineAuthResponse{
+		ShaFile: sha,
+	})
+	msg.SetTargetJobId(packet.SourceJobId)
+	a.client.Write(msg)
+
+	a.client.Emit(&MachineAuthUpdateEvent{sha})
+}
+
+func (a *Auth) handleAccountInfo(packet *Packet) {
+	body := new(CMsgClientAccountInfo)
+	packet.ReadProtoMsg(body)
+	a.client.Emit(&AccountInfoEvent{
+		PersonaName:          body.GetPersonaName(),
+		Country:              body.GetIpCountry(),
+		CountAuthedComputers: body.GetCountAuthedComputers(),
+		AccountFlags:         EAccountFlags(body.GetAccountFlags()),
+		FacebookId:           body.GetFacebookId(),
+		FacebookName:         body.GetFacebookName(),
+	})
+}

vendor/github.com/Philipp15b/go-steam/auth_events.go

@@ -0,0 +1,53 @@
+package steam
+
+import (
+	. "github.com/Philipp15b/go-steam/protocol/steamlang"
+	. "github.com/Philipp15b/go-steam/steamid"
+)
+
+type LoggedOnEvent struct {
+	Result                    EResult
+	ExtendedResult            EResult
+	OutOfGameSecsPerHeartbeat int32
+	InGameSecsPerHeartbeat    int32
+	PublicIp                  uint32
+	ServerTime                uint32
+	AccountFlags              EAccountFlags
+	ClientSteamId             SteamId `json:",string"`
+	EmailDomain               string
+	CellId                    uint32
+	CellIdPingThreshold       uint32
+	Steam2Ticket              []byte
+	UsePics                   bool
+	WebApiUserNonce           string
+	IpCountryCode             string
+	VanityUrl                 string
+	NumLoginFailuresToMigrate int32
+	NumDisconnectsToMigrate   int32
+}
+
+type LogOnFailedEvent struct {
+	Result EResult
+}
+
+type LoginKeyEvent struct {
+	UniqueId uint32
+	LoginKey string
+}
+
+type LoggedOffEvent struct {
+	Result EResult
+}
+
+type MachineAuthUpdateEvent struct {
+	Hash []byte
+}
+
+type AccountInfoEvent struct {
+	PersonaName          string
+	Country              string
+	CountAuthedComputers int32
+	AccountFlags         EAccountFlags
+	FacebookId           uint64 `json:",string"`
+	FacebookName         string
+}

vendor/github.com/Philipp15b/go-steam/client.go

@@ -0,0 +1,383 @@
+package steam
+
+import (
+	"bytes"
+	"compress/gzip"
+	"crypto/rand"
+	"encoding/binary"
+	"fmt"
+	"hash/crc32"
+	"io/ioutil"
+	"net"
+	"sync"
+	"sync/atomic"
+	"time"
+
+	"github.com/Philipp15b/go-steam/cryptoutil"
+	"github.com/Philipp15b/go-steam/netutil"
+	. "github.com/Philipp15b/go-steam/protocol"
+	. "github.com/Philipp15b/go-steam/protocol/protobuf"
+	. "github.com/Philipp15b/go-steam/protocol/steamlang"
+	. "github.com/Philipp15b/go-steam/steamid"
+)
+
+// Represents a client to the Steam network.
+// Always poll events from the channel returned by Events() or receiving messages will stop.
+// All access, unless otherwise noted, should be threadsafe.
+//
+// When a FatalErrorEvent is emitted, the connection is automatically closed. The same client can be used to reconnect.
+// Other errors don't have any effect.
+type Client struct {
+	// these need to be 64 bit aligned for sync/atomic on 32bit
+	sessionId    int32
+	_            uint32
+	steamId      uint64
+	currentJobId uint64
+
+	Auth          *Auth
+	Social        *Social
+	Web           *Web
+	Notifications *Notifications
+	Trading       *Trading
+	GC            *GameCoordinator
+
+	events        chan interface{}
+	handlers      []PacketHandler
+	handlersMutex sync.RWMutex
+
+	tempSessionKey []byte
+
+	ConnectionTimeout time.Duration
+
+	mutex     sync.RWMutex // guarding conn and writeChan
+	conn      connection
+	writeChan chan IMsg
+	writeBuf  *bytes.Buffer
+	heartbeat *time.Ticker
+}
+
+type PacketHandler interface {
+	HandlePacket(*Packet)
+}
+
+func NewClient() *Client {
+	client := &Client{
+		events:   make(chan interface{}, 3),
+		writeBuf: new(bytes.Buffer),
+	}
+	client.Auth = &Auth{client: client}
+	client.RegisterPacketHandler(client.Auth)
+	client.Social = newSocial(client)
+	client.RegisterPacketHandler(client.Social)
+	client.Web = &Web{client: client}
+	client.RegisterPacketHandler(client.Web)
+	client.Notifications = newNotifications(client)
+	client.RegisterPacketHandler(client.Notifications)
+	client.Trading = &Trading{client: client}
+	client.RegisterPacketHandler(client.Trading)
+	client.GC = newGC(client)
+	client.RegisterPacketHandler(client.GC)
+	return client
+}
+
+// Get the event channel. By convention all events are pointers, except for errors.
+// It is never closed.
+func (c *Client) Events() <-chan interface{} {
+	return c.events
+}
+
+func (c *Client) Emit(event interface{}) {
+	c.events <- event
+}
+
+// Emits a FatalErrorEvent formatted with fmt.Errorf and disconnects.
+func (c *Client) Fatalf(format string, a ...interface{}) {
+	c.Emit(FatalErrorEvent(fmt.Errorf(format, a...)))
+	c.Disconnect()
+}
+
+// Emits an error formatted with fmt.Errorf.
+func (c *Client) Errorf(format string, a ...interface{}) {
+	c.Emit(fmt.Errorf(format, a...))
+}
+
+// Registers a PacketHandler that receives all incoming packets.
+func (c *Client) RegisterPacketHandler(handler PacketHandler) {
+	c.handlersMutex.Lock()
+	defer c.handlersMutex.Unlock()
+	c.handlers = append(c.handlers, handler)
+}
+
+func (c *Client) GetNextJobId() JobId {
+	return JobId(atomic.AddUint64(&c.currentJobId, 1))
+}
+
+func (c *Client) SteamId() SteamId {
+	return SteamId(atomic.LoadUint64(&c.steamId))
+}
+
+func (c *Client) SessionId() int32 {
+	return atomic.LoadInt32(&c.sessionId)
+}
+
+func (c *Client) Connected() bool {
+	c.mutex.RLock()
+	defer c.mutex.RUnlock()
+	return c.conn != nil
+}
+
+// Connects to a random Steam server and returns its address.
+// If this client is already connected, it is disconnected first.
+// This method tries to use an address from the Steam Directory and falls
+// back to the built-in server list if the Steam Directory can't be reached.
+// If you want to connect to a specific server, use `ConnectTo`.
+func (c *Client) Connect() *netutil.PortAddr {
+	var server *netutil.PortAddr
+	if steamDirectoryCache.IsInitialized() {
+		server = steamDirectoryCache.GetRandomCM()
+	} else {
+		server = GetRandomCM()
+	}
+	c.ConnectTo(server)
+	return server
+}
+
+// Connects to a specific server.
+// You may want to use one of the `GetRandom*CM()` functions in this package.
+// If this client is already connected, it is disconnected first.
+func (c *Client) ConnectTo(addr *netutil.PortAddr) {
+	c.ConnectToBind(addr, nil)
+}
+
+// Connects to a specific server, and binds to a specified local IP
+// If this client is already connected, it is disconnected first.
+func (c *Client) ConnectToBind(addr *netutil.PortAddr, local *net.TCPAddr) {
+	c.Disconnect()
+
+	conn, err := dialTCP(local, addr.ToTCPAddr())
+	if err != nil {
+		c.Fatalf("Connect failed: %v", err)
+		return
+	}
+	c.conn = conn
+	c.writeChan = make(chan IMsg, 5)
+
+	go c.readLoop()
+	go c.writeLoop()
+}
+
+func (c *Client) Disconnect() {
+	c.mutex.Lock()
+	defer c.mutex.Unlock()
+
+	if c.conn == nil {
+		return
+	}
+
+	c.conn.Close()
+	c.conn = nil
+	if c.heartbeat != nil {
+		c.heartbeat.Stop()
+	}
+	close(c.writeChan)
+	c.Emit(&DisconnectedEvent{})
+
+}
+
+// Adds a message to the send queue. Modifications to the given message after
+// writing are not allowed (possible race conditions).
+//
+// Writes to this client when not connected are ignored.
+func (c *Client) Write(msg IMsg) {
+	if cm, ok := msg.(IClientMsg); ok {
+		cm.SetSessionId(c.SessionId())
+		cm.SetSteamId(c.SteamId())
+	}
+	c.mutex.RLock()
+	defer c.mutex.RUnlock()
+	if c.conn == nil {
+		return
+	}
+	c.writeChan <- msg
+}
+
+func (c *Client) readLoop() {
+	for {
+		// This *should* be atomic on most platforms, but the Go spec doesn't guarantee it
+		c.mutex.RLock()
+		conn := c.conn
+		c.mutex.RUnlock()
+		if conn == nil {
+			return
+		}
+		packet, err := conn.Read()
+
+		if err != nil {
+			c.Fatalf("Error reading from the connection: %v", err)
+			return
+		}
+		c.handlePacket(packet)
+	}
+}
+
+func (c *Client) writeLoop() {
+	for {
+		c.mutex.RLock()
+		conn := c.conn
+		c.mutex.RUnlock()
+		if conn == nil {
+			return
+		}
+
+		msg, ok := <-c.writeChan
+		if !ok {
+			return
+		}
+
+		err := msg.Serialize(c.writeBuf)
+		if err != nil {
+			c.writeBuf.Reset()
+			c.Fatalf("Error serializing message %v: %v", msg, err)
+			return
+		}
+
+		err = conn.Write(c.writeBuf.Bytes())
+
+		c.writeBuf.Reset()
+
+		if err != nil {
+			c.Fatalf("Error writing message %v: %v", msg, err)
+			return
+		}
+	}
+}
+
+func (c *Client) heartbeatLoop(seconds time.Duration) {
+	if c.heartbeat != nil {
+		c.heartbeat.Stop()
+	}
+	c.heartbeat = time.NewTicker(seconds * time.Second)
+	for {
+		_, ok := <-c.heartbeat.C
+		if !ok {
+			break
+		}
+		c.Write(NewClientMsgProtobuf(EMsg_ClientHeartBeat, new(CMsgClientHeartBeat)))
+	}
+	c.heartbeat = nil
+}
+
+func (c *Client) handlePacket(packet *Packet) {
+	switch packet.EMsg {
+	case EMsg_ChannelEncryptRequest:
+		c.handleChannelEncryptRequest(packet)
+	case EMsg_ChannelEncryptResult:
+		c.handleChannelEncryptResult(packet)
+	case EMsg_Multi:
+		c.handleMulti(packet)
+	case EMsg_ClientCMList:
+		c.handleClientCMList(packet)
+	}
+
+	c.handlersMutex.RLock()
+	defer c.handlersMutex.RUnlock()
+	for _, handler := range c.handlers {
+		handler.HandlePacket(packet)
+	}
+}
+
+func (c *Client) handleChannelEncryptRequest(packet *Packet) {
+	body := NewMsgChannelEncryptRequest()
+	packet.ReadMsg(body)
+
+	if body.Universe != EUniverse_Public {
+		c.Fatalf("Invalid univserse %v!", body.Universe)
+	}
+
+	c.tempSessionKey = make([]byte, 32)
+	rand.Read(c.tempSessionKey)
+	encryptedKey := cryptoutil.RSAEncrypt(GetPublicKey(EUniverse_Public), c.tempSessionKey)
+
+	payload := new(bytes.Buffer)
+	payload.Write(encryptedKey)
+	binary.Write(payload, binary.LittleEndian, crc32.ChecksumIEEE(encryptedKey))
+	payload.WriteByte(0)
+	payload.WriteByte(0)
+	payload.WriteByte(0)
+	payload.WriteByte(0)
+
+	c.Write(NewMsg(NewMsgChannelEncryptResponse(), payload.Bytes()))
+}
+
+func (c *Client) handleChannelEncryptResult(packet *Packet) {
+	body := NewMsgChannelEncryptResult()
+	packet.ReadMsg(body)
+
+	if body.Result != EResult_OK {
+		c.Fatalf("Encryption failed: %v", body.Result)
+		return
+	}
+	c.conn.SetEncryptionKey(c.tempSessionKey)
+	c.tempSessionKey = nil
+
+	c.Emit(&ConnectedEvent{})
+}
+
+func (c *Client) handleMulti(packet *Packet) {
+	body := new(CMsgMulti)
+	packet.ReadProtoMsg(body)
+
+	payload := body.GetMessageBody()
+
+	if body.GetSizeUnzipped() > 0 {
+		r, err := gzip.NewReader(bytes.NewReader(payload))
+		if err != nil {
+			c.Errorf("handleMulti: Error while decompressing: %v", err)
+			return
+		}
+
+		payload, err = ioutil.ReadAll(r)
+		if err != nil {
+			c.Errorf("handleMulti: Error while decompressing: %v", err)
+			return
+		}
+	}
+
+	pr := bytes.NewReader(payload)
+	for pr.Len() > 0 {
+		var length uint32
+		binary.Read(pr, binary.LittleEndian, &length)
+		packetData := make([]byte, length)
+		pr.Read(packetData)
+		p, err := NewPacket(packetData)
+		if err != nil {
+			c.Errorf("Error reading packet in Multi msg %v: %v", packet, err)
+			continue
+		}
+		c.handlePacket(p)
+	}
+}
+
+func (c *Client) handleClientCMList(packet *Packet) {
+	body := new(CMsgClientCMList)
+	packet.ReadProtoMsg(body)
+
+	l := make([]*netutil.PortAddr, 0)
+	for i, ip := range body.GetCmAddresses() {
+		l = append(l, &netutil.PortAddr{
+			readIp(ip),
+			uint16(body.GetCmPorts()[i]),
+		})
+	}
+
+	c.Emit(&ClientCMListEvent{l})
+}
+
+func readIp(ip uint32) net.IP {
+	r := make(net.IP, 4)
+	r[3] = byte(ip)
+	r[2] = byte(ip >> 8)
+	r[1] = byte(ip >> 16)
+	r[0] = byte(ip >> 24)
+	return r
+}

vendor/github.com/Philipp15b/go-steam/client_events.go

@@ -0,0 +1,20 @@
+package steam
+
+import (
+	"github.com/Philipp15b/go-steam/netutil"
+)
+
+// When this event is emitted by the Client, the connection is automatically closed.
+// This may be caused by a network error, for example.
+type FatalErrorEvent error
+
+type ConnectedEvent struct{}
+
+type DisconnectedEvent struct{}
+
+// A list of connection manager addresses to connect to in the future.
+// You should always save them and then select one of these
+// instead of the builtin ones for the next connection.
+type ClientCMListEvent struct {
+	Addresses []*netutil.PortAddr
+}

vendor/github.com/Philipp15b/go-steam/community/community.go

@@ -0,0 +1,35 @@
+package community
+
+import (
+	"net/http"
+	"net/http/cookiejar"
+	"net/url"
+)
+
+const cookiePath = "https://steamcommunity.com/"
+
+func SetCookies(client *http.Client, sessionId, steamLogin, steamLoginSecure string) {
+	if client.Jar == nil {
+		client.Jar, _ = cookiejar.New(new(cookiejar.Options))
+	}
+	base, err := url.Parse(cookiePath)
+	if err != nil {
+		panic(err)
+	}
+	client.Jar.SetCookies(base, []*http.Cookie{
+		// It seems that, for some reason, Steam tries to URL-decode the cookie.
+		&http.Cookie{
+			Name:  "sessionid",
+			Value: url.QueryEscape(sessionId),
+		},
+		// steamLogin is already URL-encoded.
+		&http.Cookie{
+			Name:  "steamLogin",
+			Value: steamLogin,
+		},
+		&http.Cookie{
+			Name:  "steamLoginSecure",
+			Value: steamLoginSecure,
+		},
+	})
+}

vendor/github.com/Philipp15b/go-steam/connection.go

@@ -0,0 +1,127 @@
+package steam
+
+import (
+	"crypto/aes"
+	"crypto/cipher"
+	"encoding/binary"
+	"fmt"
+	"io"
+	"net"
+	"sync"
+
+	"github.com/Philipp15b/go-steam/cryptoutil"
+	. "github.com/Philipp15b/go-steam/protocol"
+)
+
+type connection interface {
+	Read() (*Packet, error)
+	Write([]byte) error
+	Close() error
+	SetEncryptionKey([]byte)
+	IsEncrypted() bool
+}
+
+const tcpConnectionMagic uint32 = 0x31305456 // "VT01"
+
+type tcpConnection struct {
+	conn        *net.TCPConn
+	ciph        cipher.Block
+	cipherMutex sync.RWMutex
+}
+
+func dialTCP(laddr, raddr *net.TCPAddr) (*tcpConnection, error) {
+	conn, err := net.DialTCP("tcp", laddr, raddr)
+	if err != nil {
+		return nil, err
+	}
+
+	return &tcpConnection{
+		conn: conn,
+	}, nil
+}
+
+func (c *tcpConnection) Read() (*Packet, error) {
+	// All packets begin with a packet length
+	var packetLen uint32
+	err := binary.Read(c.conn, binary.LittleEndian, &packetLen)
+	if err != nil {
+		return nil, err
+	}
+
+	// A magic value follows for validation
+	var packetMagic uint32
+	err = binary.Read(c.conn, binary.LittleEndian, &packetMagic)
+	if err != nil {
+		return nil, err
+	}
+	if packetMagic != tcpConnectionMagic {
+		return nil, fmt.Errorf("Invalid connection magic! Expected %d, got %d!", tcpConnectionMagic, packetMagic)
+	}
+
+	buf := make([]byte, packetLen, packetLen)
+	_, err = io.ReadFull(c.conn, buf)
+	if err == io.ErrUnexpectedEOF {
+		return nil, io.EOF
+	}
+	if err != nil {
+		return nil, err
+	}
+
+	// Packets after ChannelEncryptResult are encrypted
+	c.cipherMutex.RLock()
+	if c.ciph != nil {
+		buf = cryptoutil.SymmetricDecrypt(c.ciph, buf)
+	}
+	c.cipherMutex.RUnlock()
+
+	return NewPacket(buf)
+}
+
+// Writes a message. This may only be used by one goroutine at a time.
+func (c *tcpConnection) Write(message []byte) error {
+	c.cipherMutex.RLock()
+	if c.ciph != nil {
+		message = cryptoutil.SymmetricEncrypt(c.ciph, message)
+	}
+	c.cipherMutex.RUnlock()
+
+	err := binary.Write(c.conn, binary.LittleEndian, uint32(len(message)))
+	if err != nil {
+		return err
+	}
+	err = binary.Write(c.conn, binary.LittleEndian, tcpConnectionMagic)
+	if err != nil {
+		return err
+	}
+
+	_, err = c.conn.Write(message)
+	return err
+}
+
+func (c *tcpConnection) Close() error {
+	return c.conn.Close()
+}
+
+func (c *tcpConnection) SetEncryptionKey(key []byte) {
+	c.cipherMutex.Lock()
+	defer c.cipherMutex.Unlock()
+	if key == nil {
+		c.ciph = nil
+		return
+	}
+	if len(key) != 32 {
+		panic("Connection AES key is not 32 bytes long!")
+	}
+
+	var err error
+	c.ciph, err = aes.NewCipher(key)
+	if err != nil {
+		panic(err)
+	}
+}
+
+func (c *tcpConnection) IsEncrypted() bool {
+	c.cipherMutex.RLock()
+	defer c.cipherMutex.RUnlock()
+	return c.ciph != nil
+}

vendor/github.com/Philipp15b/go-steam/cryptoutil/cryptoutil.go

@@ -0,0 +1,38 @@
+package cryptoutil
+
+import (
+	"crypto/aes"
+	"crypto/cipher"
+	"crypto/rand"
+)
+
+// Performs an encryption using AES/CBC/PKCS7
+// with a random IV prepended using AES/ECB/None.
+func SymmetricEncrypt(ciph cipher.Block, src []byte) []byte {
+	// get a random IV and ECB encrypt it
+	iv := make([]byte, aes.BlockSize, aes.BlockSize)
+	_, err := rand.Read(iv)
+	if err != nil {
+		panic(err)
+	}
+	encryptedIv := make([]byte, aes.BlockSize, aes.BlockSize)
+	newECBEncrypter(ciph).CryptBlocks(encryptedIv, iv)
+
+	// pad it, copy the IV to the first 16 bytes and encrypt the rest with CBC
+	encrypted := padPKCS7WithIV(src)
+	copy(encrypted, encryptedIv)
+	cipher.NewCBCEncrypter(ciph, iv).CryptBlocks(encrypted[aes.BlockSize:], encrypted[aes.BlockSize:])
+	return encrypted
+}
+
+// Decrypts data from the reader using AES/CBC/PKCS7 with an IV
+// prepended using AES/ECB/None. The src slice may not be used anymore.
+func SymmetricDecrypt(ciph cipher.Block, src []byte) []byte {
+	iv := src[:aes.BlockSize]
+	newECBDecrypter(ciph).CryptBlocks(iv, iv)
+
+	data := src[aes.BlockSize:]
+	cipher.NewCBCDecrypter(ciph, iv).CryptBlocks(data, data)
+
+	return unpadPKCS7(data)
+}

vendor/github.com/Philipp15b/go-steam/cryptoutil/ecb.go

@@ -0,0 +1,68 @@
+package cryptoutil
+
+import (
+	"crypto/cipher"
+)
+
+// From this code review: https://codereview.appspot.com/7860047/
+// by fasmat for the Go crypto/cipher package
+
+type ecb struct {
+	b         cipher.Block
+	blockSize int
+}
+
+func newECB(b cipher.Block) *ecb {
+	return &ecb{
+		b:         b,
+		blockSize: b.BlockSize(),
+	}
+}
+
+type ecbEncrypter ecb
+
+// NewECBEncrypter returns a BlockMode which encrypts in electronic code book
+// mode, using the given Block.
+func newECBEncrypter(b cipher.Block) cipher.BlockMode {
+	return (*ecbEncrypter)(newECB(b))
+}
+
+func (x *ecbEncrypter) BlockSize() int { return x.blockSize }
+
+func (x *ecbEncrypter) CryptBlocks(dst, src []byte) {
+	if len(src)%x.blockSize != 0 {
+		panic("cryptoutil/ecb: input not full blocks")
+	}
+	if len(dst) < len(src) {
+		panic("cryptoutil/ecb: output smaller than input")
+	}
+	for len(src) > 0 {
+		x.b.Encrypt(dst, src[:x.blockSize])
+		src = src[x.blockSize:]
+		dst = dst[x.blockSize:]
+	}
+}
+
+type ecbDecrypter ecb
+
+// newECBDecrypter returns a BlockMode which decrypts in electronic code book
+// mode, using the given Block.
+func newECBDecrypter(b cipher.Block) cipher.BlockMode {
+	return (*ecbDecrypter)(newECB(b))
+}
+
+func (x *ecbDecrypter) BlockSize() int { return x.blockSize }
+
+func (x *ecbDecrypter) CryptBlocks(dst, src []byte) {
+	if len(src)%x.blockSize != 0 {
+		panic("cryptoutil/ecb: input not full blocks")
+	}
+	if len(dst) < len(src) {
+		panic("cryptoutil/ecb: output smaller than input")
+	}
+	for len(src) > 0 {
+		x.b.Decrypt(dst, src[:x.blockSize])
+		src = src[x.blockSize:]
+		dst = dst[x.blockSize:]
+	}
+}

vendor/github.com/Philipp15b/go-steam/cryptoutil/pkcs7.go

@@ -0,0 +1,25 @@
+package cryptoutil
+
+import (
+	"crypto/aes"
+)
+
+// Returns a new byte array padded with PKCS7 and prepended
+// with empty space of the AES block size (16 bytes) for the IV.
+func padPKCS7WithIV(src []byte) []byte {
+	missing := aes.BlockSize - (len(src) % aes.BlockSize)
+	newSize := len(src) + aes.BlockSize + missing
+	dest := make([]byte, newSize, newSize)
+	copy(dest[aes.BlockSize:], src)
+
+	padding := byte(missing)
+	for i := newSize - missing; i < newSize; i++ {
+		dest[i] = padding
+	}
+	return dest
+}
+
+func unpadPKCS7(src []byte) []byte {
+	padLen := src[len(src)-1]
+	return src[:len(src)-int(padLen)]
+}

vendor/github.com/Philipp15b/go-steam/cryptoutil/rsa.go

@@ -0,0 +1,31 @@
+package cryptoutil
+
+import (
+	"crypto/rand"
+	"crypto/rsa"
+	"crypto/sha1"
+	"crypto/x509"
+	"errors"
+)
+
+// Parses a DER encoded RSA public key
+func ParseASN1RSAPublicKey(derBytes []byte) (*rsa.PublicKey, error) {
+	key, err := x509.ParsePKIXPublicKey(derBytes)
+	if err != nil {
+		return nil, err
+	}
+	pubKey, ok := key.(*rsa.PublicKey)
+	if !ok {
+		return nil, errors.New("not an RSA public key")
+	}
+	return pubKey, nil
+}
+
+// Encrypts a message with the given public key using RSA-OAEP and the sha1 hash function.
+func RSAEncrypt(pub *rsa.PublicKey, msg []byte) []byte {
+	b, err := rsa.EncryptOAEP(sha1.New(), rand.Reader, pub, msg, nil)
+	if err != nil {
+		panic(err)
+	}
+	return b
+}

vendor/github.com/Philipp15b/go-steam/doc.go

@@ -0,0 +1,53 @@
+/*
+This package allows you to automate actions on Valve's Steam network. It is a Go port of SteamKit.
+
+To login, you'll have to create a new Client first. Then connect to the Steam network
+and wait for a ConnectedCallback. Then you may call the Login method in the Auth module
+with your login information. This is covered in more detail in the method's documentation. After you've
+received the LoggedOnEvent, you should set your persona state to online to receive friend lists etc.
+
+Example code
+
+You can also find a running example in the `gsbot` package.
+
+	package main
+
+	import (
+		"io/ioutil"
+		"log"
+
+		"github.com/Philipp15b/go-steam"
+		"github.com/Philipp15b/go-steam/protocol/steamlang"
+	)
+
+	func main() {
+		myLoginInfo := new(steam.LogOnDetails)
+		myLoginInfo.Username = "Your username"
+		myLoginInfo.Password = "Your password"
+
+		client := steam.NewClient()
+		client.Connect()
+		for event := range client.Events() {
+			switch e := event.(type) {
+			case *steam.ConnectedEvent:
+				client.Auth.LogOn(myLoginInfo)
+			case *steam.MachineAuthUpdateEvent:
+				ioutil.WriteFile("sentry", e.Hash, 0666)
+			case *steam.LoggedOnEvent:
+				client.Social.SetPersonaState(steamlang.EPersonaState_Online)
+			case steam.FatalErrorEvent:
+				log.Print(e)
+			case error:
+				log.Print(e)
+			}
+		}
+	}
+
+
+Events
+
+go-steam emits events that can be read via Client.Events(). Although the channel has the type interface{},
+only types from this package ending with "Event" and errors will be emitted.
+
+*/
+package steam

vendor/github.com/Philipp15b/go-steam/dota/protocol/protobuf/system.pb.go

@@ -0,0 +1,579 @@
+// Code generated by protoc-gen-go.
+// source: gcsystemmsgs.proto
+// DO NOT EDIT!
+
+package protobuf
+
+import proto "github.com/golang/protobuf/proto"
+import fmt "fmt"
+import math "math"
+
+// Reference imports to suppress errors if they are not otherwise used.
+var _ = proto.Marshal
+var _ = fmt.Errorf
+var _ = math.Inf
+
+// This is a compile-time assertion to ensure that this generated file
+// is compatible with the proto package protobuf is being compiled against.
+const _ = proto.ProtoPackageIsVersion1
+
+type EGCSystemMsg int32
+
+const (
+	EGCSystemMsg_k_EGCMsgInvalid                           EGCSystemMsg = 0
+	EGCSystemMsg_k_EGCMsgMulti                             EGCSystemMsg = 1
+	EGCSystemMsg_k_EGCMsgGenericReply                      EGCSystemMsg = 10
+	EGCSystemMsg_k_EGCMsgSystemBase                        EGCSystemMsg = 50
+	EGCSystemMsg_k_EGCMsgAchievementAwarded                EGCSystemMsg = 51
+	EGCSystemMsg_k_EGCMsgConCommand                        EGCSystemMsg = 52
+	EGCSystemMsg_k_EGCMsgStartPlaying                      EGCSystemMsg = 53
+	EGCSystemMsg_k_EGCMsgStopPlaying                       EGCSystemMsg = 54
+	EGCSystemMsg_k_EGCMsgStartGameserver                   EGCSystemMsg = 55
+	EGCSystemMsg_k_EGCMsgStopGameserver                    EGCSystemMsg = 56
+	EGCSystemMsg_k_EGCMsgWGRequest                         EGCSystemMsg = 57
+	EGCSystemMsg_k_EGCMsgWGResponse                        EGCSystemMsg = 58
+	EGCSystemMsg_k_EGCMsgGetUserGameStatsSchema            EGCSystemMsg = 59
+	EGCSystemMsg_k_EGCMsgGetUserGameStatsSchemaResponse    EGCSystemMsg = 60
+	EGCSystemMsg_k_EGCMsgGetUserStatsDEPRECATED            EGCSystemMsg = 61
+	EGCSystemMsg_k_EGCMsgGetUserStatsResponse              EGCSystemMsg = 62
+	EGCSystemMsg_k_EGCMsgAppInfoUpdated                    EGCSystemMsg = 63
+	EGCSystemMsg_k_EGCMsgValidateSession                   EGCSystemMsg = 64
+	EGCSystemMsg_k_EGCMsgValidateSessionResponse           EGCSystemMsg = 65
+	EGCSystemMsg_k_EGCMsgLookupAccountFromInput            EGCSystemMsg = 66
+	EGCSystemMsg_k_EGCMsgSendHTTPRequest                   EGCSystemMsg = 67
+	EGCSystemMsg_k_EGCMsgSendHTTPRequestResponse           EGCSystemMsg = 68
+	EGCSystemMsg_k_EGCMsgPreTestSetup                      EGCSystemMsg = 69
+	EGCSystemMsg_k_EGCMsgRecordSupportAction               EGCSystemMsg = 70
+	EGCSystemMsg_k_EGCMsgGetAccountDetails_DEPRECATED      EGCSystemMsg = 71
+	EGCSystemMsg_k_EGCMsgReceiveInterAppMessage            EGCSystemMsg = 73
+	EGCSystemMsg_k_EGCMsgFindAccounts                      EGCSystemMsg = 74
+	EGCSystemMsg_k_EGCMsgPostAlert                         EGCSystemMsg = 75
+	EGCSystemMsg_k_EGCMsgGetLicenses                       EGCSystemMsg = 76
+	EGCSystemMsg_k_EGCMsgGetUserStats                      EGCSystemMsg = 77
+	EGCSystemMsg_k_EGCMsgGetCommands                       EGCSystemMsg = 78
+	EGCSystemMsg_k_EGCMsgGetCommandsResponse               EGCSystemMsg = 79
+	EGCSystemMsg_k_EGCMsgAddFreeLicense                    EGCSystemMsg = 80
+	EGCSystemMsg_k_EGCMsgAddFreeLicenseResponse            EGCSystemMsg = 81
+	EGCSystemMsg_k_EGCMsgGetIPLocation                     EGCSystemMsg = 82
+	EGCSystemMsg_k_EGCMsgGetIPLocationResponse             EGCSystemMsg = 83
+	EGCSystemMsg_k_EGCMsgSystemStatsSchema                 EGCSystemMsg = 84
+	EGCSystemMsg_k_EGCMsgGetSystemStats                    EGCSystemMsg = 85
+	EGCSystemMsg_k_EGCMsgGetSystemStatsResponse            EGCSystemMsg = 86
+	EGCSystemMsg_k_EGCMsgSendEmail                         EGCSystemMsg = 87
+	EGCSystemMsg_k_EGCMsgSendEmailResponse                 EGCSystemMsg = 88
+	EGCSystemMsg_k_EGCMsgGetEmailTemplate                  EGCSystemMsg = 89
+	EGCSystemMsg_k_EGCMsgGetEmailTemplateResponse          EGCSystemMsg = 90
+	EGCSystemMsg_k_EGCMsgGrantGuestPass                    EGCSystemMsg = 91
+	EGCSystemMsg_k_EGCMsgGrantGuestPassResponse            EGCSystemMsg = 92
+	EGCSystemMsg_k_EGCMsgGetAccountDetails                 EGCSystemMsg = 93
+	EGCSystemMsg_k_EGCMsgGetAccountDetailsResponse         EGCSystemMsg = 94
+	EGCSystemMsg_k_EGCMsgGetPersonaNames                   EGCSystemMsg = 95
+	EGCSystemMsg_k_EGCMsgGetPersonaNamesResponse           EGCSystemMsg = 96
+	EGCSystemMsg_k_EGCMsgMultiplexMsg                      EGCSystemMsg = 97
+	EGCSystemMsg_k_EGCMsgWebAPIRegisterInterfaces          EGCSystemMsg = 101
+	EGCSystemMsg_k_EGCMsgWebAPIJobRequest                  EGCSystemMsg = 102
+	EGCSystemMsg_k_EGCMsgWebAPIJobRequestHttpResponse      EGCSystemMsg = 104
+	EGCSystemMsg_k_EGCMsgWebAPIJobRequestForwardResponse   EGCSystemMsg = 105
+	EGCSystemMsg_k_EGCMsgMemCachedGet                      EGCSystemMsg = 200
+	EGCSystemMsg_k_EGCMsgMemCachedGetResponse              EGCSystemMsg = 201
+	EGCSystemMsg_k_EGCMsgMemCachedSet                      EGCSystemMsg = 202
+	EGCSystemMsg_k_EGCMsgMemCachedDelete                   EGCSystemMsg = 203
+	EGCSystemMsg_k_EGCMsgMemCachedStats                    EGCSystemMsg = 204
+	EGCSystemMsg_k_EGCMsgMemCachedStatsResponse            EGCSystemMsg = 205
+	EGCSystemMsg_k_EGCMsgSQLStats                          EGCSystemMsg = 210
+	EGCSystemMsg_k_EGCMsgSQLStatsResponse                  EGCSystemMsg = 211
+	EGCSystemMsg_k_EGCMsgMasterSetDirectory                EGCSystemMsg = 220
+	EGCSystemMsg_k_EGCMsgMasterSetDirectoryResponse        EGCSystemMsg = 221
+	EGCSystemMsg_k_EGCMsgMasterSetWebAPIRouting            EGCSystemMsg = 222
+	EGCSystemMsg_k_EGCMsgMasterSetWebAPIRoutingResponse    EGCSystemMsg = 223
+	EGCSystemMsg_k_EGCMsgMasterSetClientMsgRouting         EGCSystemMsg = 224
+	EGCSystemMsg_k_EGCMsgMasterSetClientMsgRoutingResponse EGCSystemMsg = 225
+	EGCSystemMsg_k_EGCMsgSetOptions                        EGCSystemMsg = 226
+	EGCSystemMsg_k_EGCMsgSetOptionsResponse                EGCSystemMsg = 227
+	EGCSystemMsg_k_EGCMsgSystemBase2                       EGCSystemMsg = 500
+	EGCSystemMsg_k_EGCMsgGetPurchaseTrustStatus            EGCSystemMsg = 501
+	EGCSystemMsg_k_EGCMsgGetPurchaseTrustStatusResponse    EGCSystemMsg = 502
+	EGCSystemMsg_k_EGCMsgUpdateSession                     EGCSystemMsg = 503
+	EGCSystemMsg_k_EGCMsgGCAccountVacStatusChange          EGCSystemMsg = 504
+	EGCSystemMsg_k_EGCMsgCheckFriendship                   EGCSystemMsg = 505
+	EGCSystemMsg_k_EGCMsgCheckFriendshipResponse           EGCSystemMsg = 506
+	EGCSystemMsg_k_EGCMsgGetPartnerAccountLink             EGCSystemMsg = 507
+	EGCSystemMsg_k_EGCMsgGetPartnerAccountLinkResponse     EGCSystemMsg = 508
+	EGCSystemMsg_k_EGCMsgVSReportedSuspiciousActivity      EGCSystemMsg = 509
+	EGCSystemMsg_k_EGCMsgDPPartnerMicroTxns                EGCSystemMsg = 512
+	EGCSystemMsg_k_EGCMsgDPPartnerMicroTxnsResponse        EGCSystemMsg = 513
+	EGCSystemMsg_k_EGCMsgGetIPASN                          EGCSystemMsg = 514
+	EGCSystemMsg_k_EGCMsgGetIPASNResponse                  EGCSystemMsg = 515
+	EGCSystemMsg_k_EGCMsgGetAppFriendsList                 EGCSystemMsg = 516
+	EGCSystemMsg_k_EGCMsgGetAppFriendsListResponse         EGCSystemMsg = 517
+)
+
+var EGCSystemMsg_name = map[int32]string{
+	0:   "k_EGCMsgInvalid",
+	1:   "k_EGCMsgMulti",
+	10:  "k_EGCMsgGenericReply",
+	50:  "k_EGCMsgSystemBase",
+	51:  "k_EGCMsgAchievementAwarded",
+	52:  "k_EGCMsgConCommand",
+	53:  "k_EGCMsgStartPlaying",
+	54:  "k_EGCMsgStopPlaying",
+	55:  "k_EGCMsgStartGameserver",
+	56:  "k_EGCMsgStopGameserver",
+	57:  "k_EGCMsgWGRequest",
+	58:  "k_EGCMsgWGResponse",
+	59:  "k_EGCMsgGetUserGameStatsSchema",
+	60:  "k_EGCMsgGetUserGameStatsSchemaResponse",
+	61:  "k_EGCMsgGetUserStatsDEPRECATED",
+	62:  "k_EGCMsgGetUserStatsResponse",
+	63:  "k_EGCMsgAppInfoUpdated",
+	64:  "k_EGCMsgValidateSession",
+	65:  "k_EGCMsgValidateSessionResponse",
+	66:  "k_EGCMsgLookupAccountFromInput",
+	67:  "k_EGCMsgSendHTTPRequest",
+	68:  "k_EGCMsgSendHTTPRequestResponse",
+	69:  "k_EGCMsgPreTestSetup",
+	70:  "k_EGCMsgRecordSupportAction",
+	71:  "k_EGCMsgGetAccountDetails_DEPRECATED",
+	73:  "k_EGCMsgReceiveInterAppMessage",
+	74:  "k_EGCMsgFindAccounts",
+	75:  "k_EGCMsgPostAlert",
+	76:  "k_EGCMsgGetLicenses",
+	77:  "k_EGCMsgGetUserStats",
+	78:  "k_EGCMsgGetCommands",
+	79:  "k_EGCMsgGetCommandsResponse",
+	80:  "k_EGCMsgAddFreeLicense",
+	81:  "k_EGCMsgAddFreeLicenseResponse",
+	82:  "k_EGCMsgGetIPLocation",
+	83:  "k_EGCMsgGetIPLocationResponse",
+	84:  "k_EGCMsgSystemStatsSchema",
+	85:  "k_EGCMsgGetSystemStats",
+	86:  "k_EGCMsgGetSystemStatsResponse",
+	87:  "k_EGCMsgSendEmail",
+	88:  "k_EGCMsgSendEmailResponse",
+	89:  "k_EGCMsgGetEmailTemplate",
+	90:  "k_EGCMsgGetEmailTemplateResponse",
+	91:  "k_EGCMsgGrantGuestPass",
+	92:  "k_EGCMsgGrantGuestPassResponse",
+	93:  "k_EGCMsgGetAccountDetails",
+	94:  "k_EGCMsgGetAccountDetailsResponse",
+	95:  "k_EGCMsgGetPersonaNames",
+	96:  "k_EGCMsgGetPersonaNamesResponse",
+	97:  "k_EGCMsgMultiplexMsg",
+	101: "k_EGCMsgWebAPIRegisterInterfaces",
+	102: "k_EGCMsgWebAPIJobRequest",
+	104: "k_EGCMsgWebAPIJobRequestHttpResponse",
+	105: "k_EGCMsgWebAPIJobRequestForwardResponse",
+	200: "k_EGCMsgMemCachedGet",
+	201: "k_EGCMsgMemCachedGetResponse",
+	202: "k_EGCMsgMemCachedSet",
+	203: "k_EGCMsgMemCachedDelete",
+	204: "k_EGCMsgMemCachedStats",
+	205: "k_EGCMsgMemCachedStatsResponse",
+	210: "k_EGCMsgSQLStats",
+	211: "k_EGCMsgSQLStatsResponse",
+	220: "k_EGCMsgMasterSetDirectory",
+	221: "k_EGCMsgMasterSetDirectoryResponse",
+	222: "k_EGCMsgMasterSetWebAPIRouting",
+	223: "k_EGCMsgMasterSetWebAPIRoutingResponse",
+	224: "k_EGCMsgMasterSetClientMsgRouting",
+	225: "k_EGCMsgMasterSetClientMsgRoutingResponse",
+	226: "k_EGCMsgSetOptions",
+	227: "k_EGCMsgSetOptionsResponse",
+	500: "k_EGCMsgSystemBase2",
+	501: "k_EGCMsgGetPurchaseTrustStatus",
+	502: "k_EGCMsgGetPurchaseTrustStatusResponse",
+	503: "k_EGCMsgUpdateSession",
+	504: "k_EGCMsgGCAccountVacStatusChange",
+	505: "k_EGCMsgCheckFriendship",
+	506: "k_EGCMsgCheckFriendshipResponse",
+	507: "k_EGCMsgGetPartnerAccountLink",
+	508: "k_EGCMsgGetPartnerAccountLinkResponse",
+	509: "k_EGCMsgVSReportedSuspiciousActivity",
+	512: "k_EGCMsgDPPartnerMicroTxns",
+	513: "k_EGCMsgDPPartnerMicroTxnsResponse",
+	514: "k_EGCMsgGetIPASN",
+	515: "k_EGCMsgGetIPASNResponse",
+	516: "k_EGCMsgGetAppFriendsList",
+	517: "k_EGCMsgGetAppFriendsListResponse",
+}
+var EGCSystemMsg_value = map[string]int32{
+	"k_EGCMsgInvalid":                           0,
+	"k_EGCMsgMulti":                             1,
+	"k_EGCMsgGenericReply":                      10,
+	"k_EGCMsgSystemBase":                        50,
+	"k_EGCMsgAchievementAwarded":                51,
+	"k_EGCMsgConCommand":                        52,
+	"k_EGCMsgStartPlaying":                      53,
+	"k_EGCMsgStopPlaying":                       54,
+	"k_EGCMsgStartGameserver":                   55,
+	"k_EGCMsgStopGameserver":                    56,
+	"k_EGCMsgWGRequest":                         57,
+	"k_EGCMsgWGResponse":                        58,
+	"k_EGCMsgGetUserGameStatsSchema":            59,
+	"k_EGCMsgGetUserGameStatsSchemaResponse":    60,
+	"k_EGCMsgGetUserStatsDEPRECATED":            61,
+	"k_EGCMsgGetUserStatsResponse":              62,
+	"k_EGCMsgAppInfoUpdated":                    63,
+	"k_EGCMsgValidateSession":                   64,
+	"k_EGCMsgValidateSessionResponse":           65,
+	"k_EGCMsgLookupAccountFromInput":            66,
+	"k_EGCMsgSendHTTPRequest":                   67,
+	"k_EGCMsgSendHTTPRequestResponse":           68,
+	"k_EGCMsgPreTestSetup":                      69,
+	"k_EGCMsgRecordSupportAction":               70,
+	"k_EGCMsgGetAccountDetails_DEPRECATED":      71,
+	"k_EGCMsgReceiveInterAppMessage":            73,
+	"k_EGCMsgFindAccounts":                      74,
+	"k_EGCMsgPostAlert":                         75,
+	"k_EGCMsgGetLicenses":                       76,
+	"k_EGCMsgGetUserStats":                      77,
+	"k_EGCMsgGetCommands":                       78,
+	"k_EGCMsgGetCommandsResponse":               79,
+	"k_EGCMsgAddFreeLicense":                    80,
+	"k_EGCMsgAddFreeLicenseResponse":            81,
+	"k_EGCMsgGetIPLocation":                     82,
+	"k_EGCMsgGetIPLocationResponse":             83,
+	"k_EGCMsgSystemStatsSchema":                 84,
+	"k_EGCMsgGetSystemStats":                    85,
+	"k_EGCMsgGetSystemStatsResponse":            86,
+	"k_EGCMsgSendEmail":                         87,
+	"k_EGCMsgSendEmailResponse":                 88,
+	"k_EGCMsgGetEmailTemplate":                  89,
+	"k_EGCMsgGetEmailTemplateResponse":          90,
+	"k_EGCMsgGrantGuestPass":                    91,
+	"k_EGCMsgGrantGuestPassResponse":            92,
+	"k_EGCMsgGetAccountDetails":                 93,
+	"k_EGCMsgGetAccountDetailsResponse":         94,
+	"k_EGCMsgGetPersonaNames":                   95,
+	"k_EGCMsgGetPersonaNamesResponse":           96,
+	"k_EGCMsgMultiplexMsg":                      97,
+	"k_EGCMsgWebAPIRegisterInterfaces":          101,
+	"k_EGCMsgWebAPIJobRequest":                  102,
+	"k_EGCMsgWebAPIJobRequestHttpResponse":      104,
+	"k_EGCMsgWebAPIJobRequestForwardResponse":   105,
+	"k_EGCMsgMemCachedGet":                      200,
+	"k_EGCMsgMemCachedGetResponse":              201,
+	"k_EGCMsgMemCachedSet":                      202,
+	"k_EGCMsgMemCachedDelete":                   203,
+	"k_EGCMsgMemCachedStats":                    204,
+	"k_EGCMsgMemCachedStatsResponse":            205,
+	"k_EGCMsgSQLStats":                          210,
+	"k_EGCMsgSQLStatsResponse":                  211,
+	"k_EGCMsgMasterSetDirectory":                220,
+	"k_EGCMsgMasterSetDirectoryResponse":        221,
+	"k_EGCMsgMasterSetWebAPIRouting":            222,
+	"k_EGCMsgMasterSetWebAPIRoutingResponse":    223,
+	"k_EGCMsgMasterSetClientMsgRouting":         224,
+	"k_EGCMsgMasterSetClientMsgRoutingResponse": 225,
+	"k_EGCMsgSetOptions":                        226,
+	"k_EGCMsgSetOptionsResponse":                227,
+	"k_EGCMsgSystemBase2":                       500,
+	"k_EGCMsgGetPurchaseTrustStatus":            501,
+	"k_EGCMsgGetPurchaseTrustStatusResponse":    502,
+	"k_EGCMsgUpdateSession":                     503,
+	"k_EGCMsgGCAccountVacStatusChange":          504,
+	"k_EGCMsgCheckFriendship":                   505,
+	"k_EGCMsgCheckFriendshipResponse":           506,
+	"k_EGCMsgGetPartnerAccountLink":             507,
+	"k_EGCMsgGetPartnerAccountLinkResponse":     508,
+	"k_EGCMsgVSReportedSuspiciousActivity":      509,
+	"k_EGCMsgDPPartnerMicroTxns":                512,
+	"k_EGCMsgDPPartnerMicroTxnsResponse":        513,
+	"k_EGCMsgGetIPASN":                          514,
+	"k_EGCMsgGetIPASNResponse":                  515,
+	"k_EGCMsgGetAppFriendsList":                 516,
+	"k_EGCMsgGetAppFriendsListResponse":         517,
+}
+
+func (x EGCSystemMsg) Enum() *EGCSystemMsg {
+	p := new(EGCSystemMsg)
+	*p = x
+	return p
+}
+func (x EGCSystemMsg) String() string {
+	return proto.EnumName(EGCSystemMsg_name, int32(x))
+}
+func (x *EGCSystemMsg) UnmarshalJSON(data []byte) error {
+	value, err := proto.UnmarshalJSONEnum(EGCSystemMsg_value, data, "EGCSystemMsg")
+	if err != nil {
+		return err
+	}
+	*x = EGCSystemMsg(value)
+	return nil
+}
+func (EGCSystemMsg) EnumDescriptor() ([]byte, []int) { return system_fileDescriptor0, []int{0} }
+
+type ESOMsg int32
+
+const (
+	ESOMsg_k_ESOMsg_Create                   ESOMsg = 21
+	ESOMsg_k_ESOMsg_Update                   ESOMsg = 22
+	ESOMsg_k_ESOMsg_Destroy                  ESOMsg = 23
+	ESOMsg_k_ESOMsg_CacheSubscribed          ESOMsg = 24
+	ESOMsg_k_ESOMsg_CacheUnsubscribed        ESOMsg = 25
+	ESOMsg_k_ESOMsg_UpdateMultiple           ESOMsg = 26
+	ESOMsg_k_ESOMsg_CacheSubscriptionRefresh ESOMsg = 28
+	ESOMsg_k_ESOMsg_CacheSubscribedUpToDate  ESOMsg = 29
+)
+
+var ESOMsg_name = map[int32]string{
+	21: "k_ESOMsg_Create",
+	22: "k_ESOMsg_Update",
+	23: "k_ESOMsg_Destroy",
+	24: "k_ESOMsg_CacheSubscribed",
+	25: "k_ESOMsg_CacheUnsubscribed",
+	26: "k_ESOMsg_UpdateMultiple",
+	28: "k_ESOMsg_CacheSubscriptionRefresh",
+	29: "k_ESOMsg_CacheSubscribedUpToDate",
+}
+var ESOMsg_value = map[string]int32{
+	"k_ESOMsg_Create":                   21,
+	"k_ESOMsg_Update":                   22,
+	"k_ESOMsg_Destroy":                  23,
+	"k_ESOMsg_CacheSubscribed":          24,
+	"k_ESOMsg_CacheUnsubscribed":        25,
+	"k_ESOMsg_UpdateMultiple":           26,
+	"k_ESOMsg_CacheSubscriptionRefresh": 28,
+	"k_ESOMsg_CacheSubscribedUpToDate":  29,
+}
+
+func (x ESOMsg) Enum() *ESOMsg {
+	p := new(ESOMsg)
+	*p = x
+	return p
+}
+func (x ESOMsg) String() string {
+	return proto.EnumName(ESOMsg_name, int32(x))
+}
+func (x *ESOMsg) UnmarshalJSON(data []byte) error {
+	value, err := proto.UnmarshalJSONEnum(ESOMsg_value, data, "ESOMsg")
+	if err != nil {
+		return err
+	}
+	*x = ESOMsg(value)
+	return nil
+}
+func (ESOMsg) EnumDescriptor() ([]byte, []int) { return system_fileDescriptor0, []int{1} }
+
+type EGCBaseClientMsg int32
+
+const (
+	EGCBaseClientMsg_k_EMsgGCPingRequest            EGCBaseClientMsg = 3001
+	EGCBaseClientMsg_k_EMsgGCPingResponse           EGCBaseClientMsg = 3002
+	EGCBaseClientMsg_k_EMsgGCClientWelcome          EGCBaseClientMsg = 4004
+	EGCBaseClientMsg_k_EMsgGCServerWelcome          EGCBaseClientMsg = 4005
+	EGCBaseClientMsg_k_EMsgGCClientHello            EGCBaseClientMsg = 4006
+	EGCBaseClientMsg_k_EMsgGCServerHello            EGCBaseClientMsg = 4007
+	EGCBaseClientMsg_k_EMsgGCClientConnectionStatus EGCBaseClientMsg = 4009
+	EGCBaseClientMsg_k_EMsgGCServerConnectionStatus EGCBaseClientMsg = 4010
+)
+
+var EGCBaseClientMsg_name = map[int32]string{
+	3001: "k_EMsgGCPingRequest",
+	3002: "k_EMsgGCPingResponse",
+	4004: "k_EMsgGCClientWelcome",
+	4005: "k_EMsgGCServerWelcome",
+	4006: "k_EMsgGCClientHello",
+	4007: "k_EMsgGCServerHello",
+	4009: "k_EMsgGCClientConnectionStatus",
+	4010: "k_EMsgGCServerConnectionStatus",
+}
+var EGCBaseClientMsg_value = map[string]int32{
+	"k_EMsgGCPingRequest":            3001,
+	"k_EMsgGCPingResponse":           3002,
+	"k_EMsgGCClientWelcome":          4004,
+	"k_EMsgGCServerWelcome":          4005,
+	"k_EMsgGCClientHello":            4006,
+	"k_EMsgGCServerHello":            4007,
+	"k_EMsgGCClientConnectionStatus": 4009,
+	"k_EMsgGCServerConnectionStatus": 4010,
+}
+
+func (x EGCBaseClientMsg) Enum() *EGCBaseClientMsg {
+	p := new(EGCBaseClientMsg)
+	*p = x
+	return p
+}
+func (x EGCBaseClientMsg) String() string {
+	return proto.EnumName(EGCBaseClientMsg_name, int32(x))
+}
+func (x *EGCBaseClientMsg) UnmarshalJSON(data []byte) error {
+	value, err := proto.UnmarshalJSONEnum(EGCBaseClientMsg_value, data, "EGCBaseClientMsg")
+	if err != nil {
+		return err
+	}
+	*x = EGCBaseClientMsg(value)
+	return nil
+}
+func (EGCBaseClientMsg) EnumDescriptor() ([]byte, []int) { return system_fileDescriptor0, []int{2} }
+
+type EGCToGCMsg int32
+
+const (
+	EGCToGCMsg_k_EGCToGCMsgMasterAck                   EGCToGCMsg = 150
+	EGCToGCMsg_k_EGCToGCMsgMasterAckResponse           EGCToGCMsg = 151
+	EGCToGCMsg_k_EGCToGCMsgRouted                      EGCToGCMsg = 152
+	EGCToGCMsg_k_EGCToGCMsgRoutedReply                 EGCToGCMsg = 153
+	EGCToGCMsg_k_EMsgGCUpdateSubGCSessionInfo          EGCToGCMsg = 154
+	EGCToGCMsg_k_EMsgGCRequestSubGCSessionInfo         EGCToGCMsg = 155
+	EGCToGCMsg_k_EMsgGCRequestSubGCSessionInfoResponse EGCToGCMsg = 156
+	EGCToGCMsg_k_EGCToGCMsgMasterStartupComplete       EGCToGCMsg = 157
+	EGCToGCMsg_k_EMsgGCToGCSOCacheSubscribe            EGCToGCMsg = 158
+	EGCToGCMsg_k_EMsgGCToGCSOCacheUnsubscribe          EGCToGCMsg = 159
+	EGCToGCMsg_k_EMsgGCToGCLoadSessionSOCache          EGCToGCMsg = 160
+	EGCToGCMsg_k_EMsgGCToGCLoadSessionSOCacheResponse  EGCToGCMsg = 161
+	EGCToGCMsg_k_EMsgGCToGCUpdateSessionStats          EGCToGCMsg = 162
+)
+
+var EGCToGCMsg_name = map[int32]string{
+	150: "k_EGCToGCMsgMasterAck",
+	151: "k_EGCToGCMsgMasterAckResponse",
+	152: "k_EGCToGCMsgRouted",
+	153: "k_EGCToGCMsgRoutedReply",
+	154: "k_EMsgGCUpdateSubGCSessionInfo",
+	155: "k_EMsgGCRequestSubGCSessionInfo",
+	156: "k_EMsgGCRequestSubGCSessionInfoResponse",
+	157: "k_EGCToGCMsgMasterStartupComplete",
+	158: "k_EMsgGCToGCSOCacheSubscribe",
+	159: "k_EMsgGCToGCSOCacheUnsubscribe",
+	160: "k_EMsgGCToGCLoadSessionSOCache",
+	161: "k_EMsgGCToGCLoadSessionSOCacheResponse",
+	162: "k_EMsgGCToGCUpdateSessionStats",
+}
+var EGCToGCMsg_value = map[string]int32{
+	"k_EGCToGCMsgMasterAck":                   150,
+	"k_EGCToGCMsgMasterAckResponse":           151,
+	"k_EGCToGCMsgRouted":                      152,
+	"k_EGCToGCMsgRoutedReply":                 153,
+	"k_EMsgGCUpdateSubGCSessionInfo":          154,
+	"k_EMsgGCRequestSubGCSessionInfo":         155,
+	"k_EMsgGCRequestSubGCSessionInfoResponse": 156,
+	"k_EGCToGCMsgMasterStartupComplete":       157,
+	"k_EMsgGCToGCSOCacheSubscribe":            158,
+	"k_EMsgGCToGCSOCacheUnsubscribe":          159,
+	"k_EMsgGCToGCLoadSessionSOCache":          160,
+	"k_EMsgGCToGCLoadSessionSOCacheResponse":  161,
+	"k_EMsgGCToGCUpdateSessionStats":          162,
+}
+
+func (x EGCToGCMsg) Enum() *EGCToGCMsg {
+	p := new(EGCToGCMsg)
+	*p = x
+	return p
+}
+func (x EGCToGCMsg) String() string {
+	return proto.EnumName(EGCToGCMsg_name, int32(x))
+}
+func (x *EGCToGCMsg) UnmarshalJSON(data []byte) error {
+	value, err := proto.UnmarshalJSONEnum(EGCToGCMsg_value, data, "EGCToGCMsg")
+	if err != nil {
+		return err
+	}
+	*x = EGCToGCMsg(value)
+	return nil
+}
+func (EGCToGCMsg) EnumDescriptor() ([]byte, []int) { return system_fileDescriptor0, []int{3} }
+
+func init() {
+	proto.RegisterEnum("EGCSystemMsg", EGCSystemMsg_name, EGCSystemMsg_value)
+	proto.RegisterEnum("ESOMsg", ESOMsg_name, ESOMsg_value)
+	proto.RegisterEnum("EGCBaseClientMsg", EGCBaseClientMsg_name, EGCBaseClientMsg_value)
+	proto.RegisterEnum("EGCToGCMsg", EGCToGCMsg_name, EGCToGCMsg_value)
+}
+
+var system_fileDescriptor0 = []byte{
+	// 1475 bytes of a gzipped FileDescriptorProto
+	0x1f, 0x8b, 0x08, 0x00, 0x00, 0x09, 0x6e, 0x88, 0x02, 0xff, 0x84, 0x57, 0x59, 0x73, 0x1b, 0xc5,
+	0x13, 0xcf, 0x96, 0xfc, 0xff, 0x3f, 0x4c, 0x41, 0xd1, 0x99, 0xc4, 0x47, 0x12, 0x27, 0x4a, 0x42,
+	0x0e, 0x62, 0xa8, 0x3c, 0x84, 0xfb, 0x46, 0x91, 0x64, 0x5b, 0x41, 0x8e, 0x15, 0x4b, 0xb6, 0xb9,
+	0xcd, 0x7a, 0x35, 0xb6, 0xb6, 0x2c, 0xed, 0x2c, 0x33, 0xbb, 0x26, 0x7e, 0x0b, 0xd7, 0x57, 0xe0,
+	0xbe, 0x8b, 0xa3, 0xe0, 0x1b, 0xc0, 0x27, 0xe0, 0x7c, 0x81, 0x57, 0xee, 0x7c, 0x01, 0x1e, 0xb8,
+	0x21, 0x55, 0xf4, 0xee, 0xce, 0xce, 0xce, 0x4a, 0xb2, 0x79, 0x93, 0xe6, 0xd7, 0xdd, 0xd3, 0xdd,
+	0xd3, 0xfd, 0xeb, 0x5e, 0x42, 0xd7, 0x1d, 0xb9, 0x25, 0x03, 0xd6, 0xeb, 0xc9, 0x75, 0x79, 0xda,
+	0x17, 0x3c, 0xe0, 0x53, 0x97, 0x47, 0xc9, 0x55, 0xd5, 0x99, 0x72, 0x33, 0x3e, 0x9f, 0x93, 0xeb,
+	0x74, 0x0f, 0xb9, 0x66, 0x63, 0x05, 0x4f, 0xf0, 0x77, 0xcd, 0xdb, 0xb4, 0xbb, 0x6e, 0x1b, 0x76,
+	0xd1, 0xdd, 0xe4, 0xea, 0xf4, 0x70, 0x2e, 0xec, 0x06, 0x2e, 0x58, 0x74, 0x82, 0xec, 0x4d, 0x8f,
+	0x66, 0x98, 0xc7, 0x84, 0xeb, 0x2c, 0x30, 0xbf, 0xbb, 0x05, 0x84, 0x8e, 0x11, 0x9a, 0x22, 0x89,
+	0xd9, 0xb3, 0xb6, 0x64, 0x70, 0x86, 0x1e, 0x22, 0xfb, 0xd3, 0xf3, 0x92, 0xd3, 0x71, 0xd9, 0x26,
+	0xeb, 0x31, 0x2f, 0x28, 0x3d, 0x69, 0x8b, 0x36, 0x6b, 0xc3, 0x8d, 0xa6, 0x5e, 0x99, 0x7b, 0x65,
+	0xde, 0xeb, 0xd9, 0x5e, 0x1b, 0x6e, 0x32, 0x6f, 0x6a, 0x06, 0xb6, 0x08, 0x1a, 0x5d, 0x7b, 0xcb,
+	0xf5, 0xd6, 0xe1, 0x66, 0x3a, 0x4e, 0xf6, 0x64, 0x08, 0xf7, 0x53, 0xe0, 0x16, 0x7a, 0x80, 0x8c,
+	0xe7, 0x54, 0x66, 0xec, 0x1e, 0x93, 0x4c, 0x6c, 0x32, 0x01, 0xb7, 0xd2, 0xfd, 0x64, 0xcc, 0xd4,
+	0x32, 0xb0, 0xdb, 0xe8, 0x28, 0xd9, 0x9d, 0x62, 0xcb, 0x33, 0x0b, 0xec, 0x89, 0x90, 0xc9, 0x00,
+	0x6e, 0x37, 0x5d, 0x8b, 0x8e, 0xa5, 0xcf, 0x3d, 0x0c, 0xe9, 0x0e, 0x7a, 0x94, 0x1c, 0xca, 0x92,
+	0x10, 0x2c, 0xa2, 0x99, 0xc8, 0x1a, 0x5e, 0x19, 0xc8, 0xa6, 0xd3, 0x61, 0x3d, 0x1b, 0xee, 0xa4,
+	0x53, 0xe4, 0xc4, 0xce, 0x32, 0xda, 0xde, 0x5d, 0x43, 0xec, 0xc5, 0x72, 0x95, 0x6a, 0x63, 0xa1,
+	0x5a, 0x2e, 0xb5, 0xaa, 0x15, 0xb8, 0x9b, 0x1e, 0x26, 0x93, 0xc3, 0x64, 0xb4, 0x95, 0x7b, 0xcc,
+	0x00, 0x4b, 0xbe, 0x5f, 0xf3, 0xd6, 0xf8, 0xa2, 0xdf, 0xb6, 0x03, 0x4c, 0xf2, 0xbd, 0x66, 0x66,
+	0x96, 0xa2, 0xc7, 0xc5, 0xe3, 0x26, 0x93, 0xd2, 0xe5, 0x1e, 0xdc, 0x47, 0xaf, 0x25, 0xc5, 0x6d,
+	0x40, 0x6d, 0xbd, 0x64, 0xfa, 0x58, 0xe7, 0x7c, 0x23, 0xf4, 0x4b, 0x8e, 0xc3, 0x43, 0x2f, 0x98,
+	0x16, 0xbc, 0x57, 0xf3, 0xfc, 0x30, 0x80, 0xb3, 0xb9, 0xfc, 0x33, 0xaf, 0x3d, 0xdb, 0x6a, 0x35,
+	0xd2, 0x64, 0x96, 0xcd, 0x5b, 0xfa, 0x40, 0x7d, 0x4b, 0xc5, 0x7c, 0xf4, 0x86, 0x60, 0x2d, 0x04,
+	0x9b, 0x2c, 0x08, 0x7d, 0xa8, 0xd2, 0x22, 0x39, 0x90, 0x22, 0x0b, 0xcc, 0xe1, 0xa2, 0xdd, 0x0c,
+	0x7d, 0x9f, 0x8b, 0xa0, 0xe4, 0x04, 0x51, 0x14, 0xd3, 0xf4, 0x3a, 0x72, 0xcc, 0x48, 0x90, 0xf2,
+	0xae, 0xc2, 0x02, 0xdb, 0xed, 0xca, 0x15, 0x23, 0x95, 0x33, 0x66, 0x28, 0x68, 0x8a, 0xb9, 0x9b,
+	0xac, 0xe6, 0x05, 0x4c, 0x60, 0xd2, 0xe6, 0x30, 0x6c, 0x7b, 0x9d, 0x41, 0xcd, 0x74, 0x64, 0xda,
+	0xf5, 0xda, 0xca, 0x9c, 0x84, 0x73, 0x66, 0xad, 0x34, 0xb8, 0x0c, 0x4a, 0x5d, 0x26, 0x02, 0xb8,
+	0xdf, 0x2c, 0x4a, 0xbc, 0xbe, 0xee, 0x3a, 0x0c, 0x23, 0x92, 0x50, 0xcf, 0x77, 0x4c, 0xf6, 0x70,
+	0x30, 0xd7, 0xa7, 0xa2, 0x2a, 0x5f, 0xc2, 0x79, 0x33, 0x56, 0x03, 0xd0, 0x69, 0x9a, 0xcf, 0x3d,
+	0x75, 0xbb, 0x3d, 0x2d, 0x18, 0x53, 0x17, 0x42, 0xc3, 0x8c, 0x2e, 0x8f, 0x69, 0xfd, 0x0b, 0x74,
+	0x1f, 0x19, 0x35, 0x2e, 0xa8, 0x35, 0xea, 0xdc, 0xb1, 0xe3, 0x34, 0x2e, 0xd0, 0x23, 0xe4, 0xe0,
+	0x50, 0x48, 0x6b, 0x37, 0xe9, 0x41, 0xb2, 0x2f, 0xdf, 0xe9, 0x66, 0xe5, 0xb7, 0x4c, 0xe7, 0xd0,
+	0x82, 0x21, 0x01, 0x8b, 0x7d, 0x95, 0x6e, 0x60, 0xda, 0xfc, 0x92, 0x99, 0xe0, 0xa8, 0x50, 0xaa,
+	0x3d, 0x7c, 0x41, 0x58, 0xce, 0xdd, 0x9a, 0x1e, 0x6b, 0xad, 0x07, 0xe8, 0x24, 0x99, 0x30, 0x2c,
+	0xc7, 0x68, 0x8b, 0xf5, 0xfc, 0x2e, 0x16, 0x33, 0x3c, 0x48, 0x8f, 0x91, 0xc3, 0xdb, 0xa1, 0xda,
+	0xc6, 0x43, 0x39, 0xcf, 0x85, 0xed, 0x05, 0x33, 0x51, 0x75, 0x36, 0x6c, 0x29, 0xe1, 0xe1, 0x9c,
+	0xe7, 0x39, 0x4c, 0xeb, 0x3f, 0x62, 0xba, 0x38, 0x50, 0x82, 0xf0, 0x28, 0x3d, 0x4e, 0x8e, 0x6c,
+	0x0b, 0x6b, 0x2b, 0x8f, 0x99, 0x5d, 0x84, 0x62, 0x0d, 0x26, 0x24, 0xf7, 0xec, 0xf3, 0x11, 0x5d,
+	0xc1, 0x8a, 0xd9, 0x45, 0x7d, 0xa0, 0xb6, 0xf0, 0xb8, 0x59, 0x72, 0x31, 0x6f, 0xfb, 0x5d, 0x76,
+	0x11, 0x7f, 0x83, 0x6d, 0xe6, 0x61, 0x99, 0xad, 0x96, 0x1a, 0xb5, 0x05, 0xb6, 0xee, 0xe2, 0x23,
+	0x88, 0xb8, 0x03, 0xd6, 0x6c, 0x07, 0x2f, 0x61, 0x66, 0x2e, 0x13, 0xa9, 0x73, 0x7c, 0x35, 0x6d,
+	0xe4, 0x35, 0xb3, 0xd1, 0xfa, 0xd1, 0xd9, 0x20, 0xf0, 0xb5, 0x1f, 0x1d, 0x7a, 0x3d, 0x39, 0xb9,
+	0x9d, 0xe4, 0x34, 0x17, 0xd1, 0x04, 0xd0, 0xc2, 0x2e, 0xd6, 0x64, 0xe6, 0x34, 0xeb, 0x95, 0x6d,
+	0x2c, 0xa7, 0x36, 0x86, 0x08, 0x9f, 0x58, 0x58, 0x93, 0x93, 0xc3, 0x20, 0xad, 0xfc, 0xa9, 0x35,
+	0x54, 0x1b, 0xa9, 0x03, 0x3e, 0xb3, 0x30, 0x9a, 0xf1, 0x01, 0xa8, 0xc2, 0xba, 0x0c, 0x0b, 0xe3,
+	0x73, 0x0b, 0xb3, 0x3d, 0x36, 0xa8, 0x18, 0x57, 0xeb, 0x17, 0x16, 0x66, 0xfb, 0xd0, 0x70, 0x50,
+	0x5f, 0xfd, 0xa5, 0x85, 0xf5, 0x0a, 0xba, 0x30, 0x2f, 0xd4, 0x13, 0xdd, 0xaf, 0x2c, 0x2c, 0x86,
+	0x89, 0xfe, 0x63, 0xad, 0xf5, 0xb5, 0x85, 0x3d, 0xae, 0xc7, 0xe2, 0x9c, 0x1d, 0xbd, 0x00, 0x7a,
+	0x5b, 0x71, 0x05, 0x73, 0x02, 0x2e, 0xb6, 0xe0, 0x1b, 0x8b, 0x9e, 0x24, 0x47, 0xb7, 0x17, 0xd0,
+	0x96, 0xbe, 0xcd, 0x3b, 0x99, 0x0a, 0xaa, 0xc7, 0xe5, 0x61, 0x10, 0x4d, 0xc6, 0xef, 0x2c, 0x7c,
+	0x8a, 0x13, 0x3b, 0x0b, 0x69, 0x8b, 0xdf, 0x5b, 0xf4, 0x44, 0x56, 0xa8, 0x5a, 0xb8, 0xdc, 0x75,
+	0x71, 0x6c, 0x47, 0x94, 0xa9, 0x8c, 0xfe, 0x60, 0xd1, 0xd3, 0xe4, 0xd4, 0x7f, 0xca, 0x69, 0xbb,
+	0x3f, 0x5a, 0x48, 0x78, 0xd9, 0x8a, 0xc0, 0x82, 0x79, 0x3f, 0xe2, 0x15, 0x09, 0x3f, 0xe5, 0x92,
+	0x91, 0x01, 0x5a, 0xf3, 0x72, 0xb4, 0x76, 0xec, 0x19, 0x5c, 0x2e, 0xce, 0xc0, 0x2f, 0x05, 0x33,
+	0xfa, 0xa8, 0x21, 0x42, 0xe1, 0x74, 0x10, 0x6a, 0x89, 0x10, 0x47, 0x07, 0xe6, 0x3c, 0x94, 0xf0,
+	0x6b, 0xc1, 0x8c, 0x7e, 0xb8, 0x90, 0xbe, 0xeb, 0xb7, 0x02, 0xb2, 0x80, 0x26, 0xc7, 0x64, 0x80,
+	0xa6, 0x93, 0xf2, 0xf7, 0x02, 0xb6, 0x70, 0xc6, 0x23, 0x65, 0xd5, 0xc1, 0x4b, 0xb6, 0x93, 0x18,
+	0x29, 0x77, 0x6c, 0x0f, 0x87, 0xc7, 0x1f, 0x05, 0xb3, 0xe4, 0xca, 0x1d, 0xe6, 0x6c, 0x4c, 0x0b,
+	0x4c, 0x4a, 0x5b, 0x76, 0x5c, 0x1f, 0xfe, 0x2c, 0x60, 0x13, 0x16, 0xb7, 0x41, 0xb5, 0x1b, 0x7f,
+	0x15, 0x90, 0x70, 0x4c, 0x22, 0x6e, 0xe0, 0x3a, 0x83, 0xeb, 0x96, 0xba, 0xb2, 0xee, 0x7a, 0x1b,
+	0xf0, 0x77, 0x01, 0x97, 0x8c, 0xe3, 0x3b, 0xca, 0x68, 0x7b, 0xff, 0x14, 0xe8, 0xa9, 0xac, 0x6d,
+	0x97, 0x9a, 0xb8, 0xb4, 0xe1, 0xec, 0xc4, 0x62, 0x0e, 0xa5, 0xef, 0x3a, 0x2e, 0x0f, 0x65, 0x34,
+	0x47, 0x37, 0xdd, 0x60, 0x0b, 0xae, 0x14, 0xcc, 0xe7, 0xa8, 0x34, 0x94, 0xd5, 0x39, 0xd7, 0x11,
+	0xbc, 0x75, 0x11, 0xdf, 0xeb, 0xd2, 0x88, 0x59, 0x9b, 0x83, 0x02, 0xfa, 0xd2, 0xa7, 0x46, 0xcc,
+	0xde, 0x88, 0xa7, 0x49, 0xa9, 0x79, 0x1e, 0x9e, 0x1e, 0x31, 0x7b, 0x23, 0x3d, 0xd6, 0x5a, 0xcf,
+	0x8c, 0xe0, 0xca, 0x98, 0xe3, 0x51, 0xdf, 0x57, 0x19, 0xaa, 0x23, 0x55, 0xc1, 0xb3, 0x23, 0x66,
+	0x7d, 0x0e, 0xe0, 0xda, 0xce, 0x73, 0x23, 0x53, 0x3f, 0x5b, 0xe4, 0xff, 0xd5, 0xe6, 0x7c, 0xb6,
+	0xdf, 0xc6, 0xbf, 0x57, 0xca, 0x82, 0x45, 0x53, 0x61, 0x34, 0x77, 0x98, 0x3c, 0x35, 0x8c, 0xd1,
+	0xbd, 0xb1, 0xcb, 0xc9, 0x61, 0x05, 0x99, 0x4a, 0xf0, 0x2d, 0x18, 0x57, 0x94, 0xa8, 0xf4, 0x23,
+	0x1e, 0x68, 0x86, 0xab, 0xd2, 0x11, 0xee, 0x2a, 0xae, 0x57, 0x13, 0x6a, 0xc7, 0x35, 0xd0, 0x45,
+	0x4f, 0x66, 0xf8, 0x3e, 0x45, 0xe9, 0xe6, 0x45, 0x29, 0x2f, 0xc3, 0x7e, 0x35, 0x16, 0x06, 0x4d,
+	0xfb, 0xc9, 0xd8, 0x5d, 0x13, 0x4c, 0x76, 0x60, 0x52, 0x51, 0xf7, 0x50, 0x0f, 0x16, 0xfd, 0x16,
+	0xaf, 0x44, 0xde, 0x1f, 0x9c, 0xba, 0x62, 0x11, 0xc0, 0xcc, 0x44, 0xed, 0xa1, 0x3b, 0x51, 0x75,
+	0x4f, 0x5c, 0xb3, 0x8d, 0xb8, 0x25, 0x13, 0x2a, 0xff, 0x68, 0x5c, 0xd1, 0xa6, 0x81, 0xa8, 0xe4,
+	0x7d, 0x3c, 0xae, 0xda, 0x20, 0x86, 0x12, 0x4b, 0xcb, 0xac, 0xeb, 0xf0, 0x1e, 0x83, 0x77, 0x8a,
+	0x26, 0xd6, 0x8c, 0x77, 0xe8, 0x14, 0x7b, 0xb7, 0x68, 0x5e, 0x96, 0xe8, 0xcd, 0xb2, 0x6e, 0x97,
+	0xc3, 0x7b, 0x39, 0x24, 0xd1, 0x4a, 0x90, 0xf7, 0x8b, 0xaa, 0x89, 0x0d, 0x1d, 0xfc, 0x12, 0xf0,
+	0x58, 0xbc, 0xd9, 0xa9, 0x26, 0xfe, 0x20, 0x27, 0x94, 0xa8, 0x0f, 0x08, 0x7d, 0x58, 0x9c, 0xba,
+	0x5c, 0x20, 0x04, 0xe3, 0x6f, 0xf1, 0xb8, 0x3a, 0x74, 0x2f, 0xab, 0xff, 0x09, 0x4b, 0x95, 0x9c,
+	0x0d, 0x78, 0xde, 0xd2, 0x0d, 0xd6, 0x8f, 0xe9, 0x24, 0xbc, 0x90, 0x31, 0x96, 0x92, 0x89, 0x38,
+	0x0d, 0x1f, 0xf4, 0xc5, 0x6c, 0xa8, 0xe4, 0x80, 0xe4, 0x53, 0xe8, 0x25, 0xcb, 0x74, 0x55, 0x51,
+	0x48, 0xb8, 0x1a, 0x79, 0x1d, 0xf3, 0x48, 0xb4, 0x99, 0xc3, 0xcb, 0x96, 0xa2, 0x81, 0x58, 0x48,
+	0xbd, 0xc8, 0x80, 0xd4, 0x2b, 0x16, 0xbd, 0x21, 0x9e, 0xa1, 0x3b, 0x49, 0x69, 0x7f, 0x5f, 0xcd,
+	0x98, 0x3b, 0x17, 0x53, 0xfc, 0x2d, 0x14, 0xfa, 0xb8, 0x47, 0xfa, 0xf1, 0xd4, 0x7b, 0x2d, 0x9d,
+	0xa8, 0xb1, 0xd5, 0x48, 0xb4, 0x39, 0x9f, 0xaf, 0x28, 0x78, 0x3d, 0x17, 0x83, 0x21, 0x62, 0x14,
+	0x36, 0xbc, 0x31, 0x20, 0x54, 0xe7, 0x76, 0x5b, 0x79, 0xa6, 0xe4, 0xe1, 0xcd, 0x74, 0xf6, 0xec,
+	0x20, 0xa4, 0x23, 0x78, 0x6b, 0xc0, 0x62, 0x8e, 0x81, 0x93, 0xd9, 0xfa, 0xb6, 0x75, 0xf6, 0x7f,
+	0xb3, 0xd6, 0x25, 0x6b, 0xd7, 0xbf, 0x01, 0x00, 0x00, 0xff, 0xff, 0x05, 0xab, 0xaf, 0x14, 0xda,
+	0x0e, 0x00, 0x00,
+}

vendor/github.com/Philipp15b/go-steam/economy/inventory/inventory.go

@@ -0,0 +1,188 @@
+/*
+Includes inventory types as used in the trade package
+*/
+package inventory
+
+import (
+	"bytes"
+	"encoding/json"
+	"fmt"
+	"github.com/Philipp15b/go-steam/jsont"
+	"strconv"
+)
+
+type GenericInventory map[uint32]map[uint64]*Inventory
+
+func NewGenericInventory() GenericInventory {
+	iMap := make(map[uint32]map[uint64]*Inventory)
+	return GenericInventory(iMap)
+}
+
+// Get inventory for specified AppId and ContextId
+func (i *GenericInventory) Get(appId uint32, contextId uint64) (*Inventory, error) {
+	iMap := (map[uint32]map[uint64]*Inventory)(*i)
+	iMap2, ok := iMap[appId]
+	if !ok {
+		return nil, fmt.Errorf("inventory for specified appId not found")
+	}
+	inv, ok := iMap2[contextId]
+	if !ok {
+		return nil, fmt.Errorf("inventory for specified contextId not found")
+	}
+	return inv, nil
+}
+
+func (i *GenericInventory) Add(appId uint32, contextId uint64, inv *Inventory) {
+	iMap := (map[uint32]map[uint64]*Inventory)(*i)
+	iMap2, ok := iMap[appId]
+	if !ok {
+		iMap2 = make(map[uint64]*Inventory)
+		iMap[appId] = iMap2
+	}
+	iMap2[contextId] = inv
+}
+
+type Inventory struct {
+	Items        Items        `json:"rgInventory"`
+	Currencies   Currencies   `json:"rgCurrency"`
+	Descriptions Descriptions `json:"rgDescriptions"`
+	AppInfo      *AppInfo     `json:"rgAppInfo"`
+}
+
+// Items key is an AssetId
+type Items map[string]*Item
+
+func (i *Items) ToMap() map[string]*Item {
+	return (map[string]*Item)(*i)
+}
+
+func (i *Items) Get(assetId uint64) (*Item, error) {
+	iMap := (map[string]*Item)(*i)
+	if item, ok := iMap[strconv.FormatUint(assetId, 10)]; ok {
+		return item, nil
+	}
+	return nil, fmt.Errorf("item not found")
+}
+
+func (i *Items) UnmarshalJSON(data []byte) error {
+	if bytes.Equal(data, []byte("[]")) {
+		return nil
+	}
+	return json.Unmarshal(data, (*map[string]*Item)(i))
+}
+
+type Currencies map[string]*Currency
+
+func (c *Currencies) ToMap() map[string]*Currency {
+	return (map[string]*Currency)(*c)
+}
+
+func (c *Currencies) UnmarshalJSON(data []byte) error {
+	if bytes.Equal(data, []byte("[]")) {
+		return nil
+	}
+	return json.Unmarshal(data, (*map[string]*Currency)(c))
+}
+
+// Descriptions key format is %d_%d, first %d is ClassId, second is InstanceId
+type Descriptions map[string]*Description
+
+func (d *Descriptions) ToMap() map[string]*Description {
+	return (map[string]*Description)(*d)
+}
+
+func (d *Descriptions) Get(classId uint64, instanceId uint64) (*Description, error) {
+	dMap := (map[string]*Description)(*d)
+	descId := fmt.Sprintf("%v_%v", classId, instanceId)
+	if desc, ok := dMap[descId]; ok {
+		return desc, nil
+	}
+	return nil, fmt.Errorf("description not found")
+}
+
+func (d *Descriptions) UnmarshalJSON(data []byte) error {
+	if bytes.Equal(data, []byte("[]")) {
+		return nil
+	}
+	return json.Unmarshal(data, (*map[string]*Description)(d))
+}
+
+type Item struct {
+	Id         uint64 `json:",string"`
+	ClassId    uint64 `json:",string"`
+	InstanceId uint64 `json:",string"`
+	Amount     uint64 `json:",string"`
+	Pos        uint32
+}
+
+type Currency struct {
+	Id         uint64 `json:",string"`
+	ClassId    uint64 `json:",string"`
+	IsCurrency bool   `json:"is_currency"`
+	Pos        uint32
+}
+
+type Description struct {
+	AppId      uint32 `json:",string"`
+	ClassId    uint64 `json:",string"`
+	InstanceId uint64 `json:",string"`
+
+	IconUrl      string `json:"icon_url"`
+	IconUrlLarge string `json:"icon_url_large"`
+	IconDragUrl  string `json:"icon_drag_url"`
+
+	Name           string
+	MarketName     string `json:"market_name"`
+	MarketHashName string `json:"market_hash_name"`
+
+	// Colors in hex, for example `B2B2B2`
+	NameColor       string `json:"name_color"`
+	BackgroundColor string `json:"background_color"`
+
+	Type string
+
+	Tradable                  jsont.UintBool
+	Marketable                jsont.UintBool
+	Commodity                 jsont.UintBool
+	MarketTradableRestriction uint32 `json:"market_tradable_restriction,string"`
+
+	Descriptions DescriptionLines
+	Actions      []*Action
+	// Application-specific data, like "def_index" and "quality" for TF2
+	AppData map[string]string
+	Tags    []*Tag
+}
+
+type DescriptionLines []*DescriptionLine
+
+func (d *DescriptionLines) UnmarshalJSON(data []byte) error {
+	if bytes.Equal(data, []byte(`""`)) {
+		return nil
+	}
+	return json.Unmarshal(data, (*[]*DescriptionLine)(d))
+}
+
+type DescriptionLine struct {
+	Value string
+	Type  *string // Is `html` for HTML descriptions
+	Color *string
+}
+
+type Action struct {
+	Name string
+	Link string
+}
+
+type AppInfo struct {
+	AppId uint32
+	Name  string
+	Icon  string
+	Link  string
+}
+
+type Tag struct {
+	InternalName string `json:internal_name`
+	Name         string
+	Category     string
+	CategoryName string `json:category_name`
+}

vendor/github.com/Philipp15b/go-steam/economy/inventory/inventory_apps.go

@@ -0,0 +1,79 @@
+package inventory
+
+import (
+	"encoding/json"
+	"fmt"
+	"github.com/Philipp15b/go-steam/steamid"
+	"io/ioutil"
+	"net/http"
+	"regexp"
+	"strconv"
+)
+
+type InventoryApps map[string]*InventoryApp
+
+func (i *InventoryApps) Get(appId uint32) (*InventoryApp, error) {
+	iMap := (map[string]*InventoryApp)(*i)
+	if inventoryApp, ok := iMap[strconv.FormatUint(uint64(appId), 10)]; ok {
+		return inventoryApp, nil
+	}
+	return nil, fmt.Errorf("inventory app not found")
+}
+
+func (i *InventoryApps) ToMap() map[string]*InventoryApp {
+	return (map[string]*InventoryApp)(*i)
+}
+
+type InventoryApp struct {
+	AppId            uint32
+	Name             string
+	Icon             string
+	Link             string
+	AssetCount       uint32   `json:"asset_count"`
+	InventoryLogo    string   `json:"inventory_logo"`
+	TradePermissions string   `json:"trade_permissions"`
+	Contexts         Contexts `json:"rgContexts"`
+}
+
+type Contexts map[string]*Context
+
+func (c *Contexts) Get(contextId uint64) (*Context, error) {
+	cMap := (map[string]*Context)(*c)
+	if context, ok := cMap[strconv.FormatUint(contextId, 10)]; ok {
+		return context, nil
+	}
+	return nil, fmt.Errorf("context not found")
+}
+
+func (c *Contexts) ToMap() map[string]*Context {
+	return (map[string]*Context)(*c)
+}
+
+type Context struct {
+	ContextId  uint64 `json:"id,string"`
+	AssetCount uint32 `json:"asset_count"`
+	Name       string
+}
+
+func GetInventoryApps(client *http.Client, steamId steamid.SteamId) (InventoryApps, error) {
+	resp, err := http.Get("http://steamcommunity.com/profiles/" + steamId.ToString() + "/inventory/")
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+	respBody, err := ioutil.ReadAll(resp.Body)
+	if err != nil {
+		return nil, err
+	}
+	reg := regexp.MustCompile("var g_rgAppContextData = (.*?);")
+	inventoryAppsMatches := reg.FindSubmatch(respBody)
+	if inventoryAppsMatches == nil {
+		return nil, fmt.Errorf("profile inventory not found in steam response")
+	}
+	var inventoryApps InventoryApps
+	if err = json.Unmarshal(inventoryAppsMatches[1], &inventoryApps); err != nil {
+		return nil, err
+	}
+
+	return inventoryApps, nil
+}

vendor/github.com/Philipp15b/go-steam/economy/inventory/own.go

@@ -0,0 +1,28 @@
+package inventory
+
+import (
+	"fmt"
+	"net/http"
+	"strconv"
+)
+
+func GetPartialOwnInventory(client *http.Client, contextId uint64, appId uint32, start *uint) (*PartialInventory, error) {
+	// TODO: the "trading" parameter can be left off to return non-tradable items too
+	url := fmt.Sprintf("http://steamcommunity.com/my/inventory/json/%d/%d?trading=1", appId, contextId)
+	if start != nil {
+		url += "&start=" + strconv.FormatUint(uint64(*start), 10)
+	}
+	req, err := http.NewRequest("GET", url, nil)
+	if err != nil {
+		panic(err)
+	}
+	return DoInventoryRequest(client, req)
+}
+
+func GetOwnInventory(client *http.Client, contextId uint64, appId uint32) (*Inventory, error) {
+	return GetFullInventory(func() (*PartialInventory, error) {
+		return GetPartialOwnInventory(client, contextId, appId, nil)
+	}, func(start uint) (*PartialInventory, error) {
+		return GetPartialOwnInventory(client, contextId, appId, &start)
+	})
+}

vendor/github.com/Philipp15b/go-steam/economy/inventory/partial.go

@@ -0,0 +1,91 @@
+package inventory
+
+import (
+	"bytes"
+	"encoding/json"
+	"errors"
+	"net/http"
+)
+
+// A partial inventory as sent by the Steam API.
+type PartialInventory struct {
+	Success bool
+	Error   string
+	Inventory
+	More      bool
+	MoreStart MoreStart `json:"more_start"`
+}
+
+type MoreStart uint
+
+func (m *MoreStart) UnmarshalJSON(data []byte) error {
+	if bytes.Equal(data, []byte("false")) {
+		return nil
+	}
+	return json.Unmarshal(data, (*uint)(m))
+}
+
+func DoInventoryRequest(client *http.Client, req *http.Request) (*PartialInventory, error) {
+	resp, err := client.Do(req)
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+
+	inv := new(PartialInventory)
+	err = json.NewDecoder(resp.Body).Decode(inv)
+	if err != nil {
+		return nil, err
+	}
+	return inv, nil
+}
+
+func GetFullInventory(getFirst func() (*PartialInventory, error), getNext func(start uint) (*PartialInventory, error)) (*Inventory, error) {
+	first, err := getFirst()
+	if err != nil {
+		return nil, err
+	}
+	if !first.Success {
+		return nil, errors.New("GetFullInventory API call failed: " + first.Error)
+	}
+
+	result := &first.Inventory
+	var next *PartialInventory
+	for latest := first; latest.More; latest = next {
+		next, err := getNext(uint(latest.MoreStart))
+		if err != nil {
+			return nil, err
+		}
+		if !next.Success {
+			return nil, errors.New("GetFullInventory API call failed: " + next.Error)
+		}
+
+		result = Merge(result, &next.Inventory)
+	}
+
+	return result, nil
+}
+
+// Merges the given Inventory into a single Inventory.
+// The given slice must have at least one element. The first element of the slice is used
+// and modified.
+func Merge(p ...*Inventory) *Inventory {
+	inv := p[0]
+	for idx, i := range p {
+		if idx == 0 {
+			continue
+		}
+
+		for key, value := range i.Items {
+			inv.Items[key] = value
+		}
+		for key, value := range i.Descriptions {
+			inv.Descriptions[key] = value
+		}
+		for key, value := range i.Currencies {
+			inv.Currencies[key] = value
+		}
+	}
+
+	return inv
+}

vendor/github.com/Philipp15b/go-steam/gamecoordinator.go

@@ -0,0 +1,79 @@
+package steam
+
+import (
+	"bytes"
+	. "github.com/Philipp15b/go-steam/protocol"
+	. "github.com/Philipp15b/go-steam/protocol/gamecoordinator"
+	. "github.com/Philipp15b/go-steam/protocol/protobuf"
+	. "github.com/Philipp15b/go-steam/protocol/steamlang"
+	"github.com/golang/protobuf/proto"
+)
+
+type GameCoordinator struct {
+	client   *Client
+	handlers []GCPacketHandler
+}
+
+func newGC(client *Client) *GameCoordinator {
+	return &GameCoordinator{
+		client:   client,
+		handlers: make([]GCPacketHandler, 0),
+	}
+}
+
+type GCPacketHandler interface {
+	HandleGCPacket(*GCPacket)
+}
+
+func (g *GameCoordinator) RegisterPacketHandler(handler GCPacketHandler) {
+	g.handlers = append(g.handlers, handler)
+}
+
+func (g *GameCoordinator) HandlePacket(packet *Packet) {
+	if packet.EMsg != EMsg_ClientFromGC {
+		return
+	}
+
+	msg := new(CMsgGCClient)
+	packet.ReadProtoMsg(msg)
+
+	p, err := NewGCPacket(msg)
+	if err != nil {
+		g.client.Errorf("Error reading GC message: %v", err)
+		return
+	}
+
+	for _, handler := range g.handlers {
+		handler.HandleGCPacket(p)
+	}
+}
+
+func (g *GameCoordinator) Write(msg IGCMsg) {
+	buf := new(bytes.Buffer)
+	msg.Serialize(buf)
+
+	msgType := msg.GetMsgType()
+	if msg.IsProto() {
+		msgType = msgType | 0x80000000 // mask with protoMask
+	}
+
+	g.client.Write(NewClientMsgProtobuf(EMsg_ClientToGC, &CMsgGCClient{
+		Msgtype: proto.Uint32(msgType),
+		Appid:   proto.Uint32(msg.GetAppId()),
+		Payload: buf.Bytes(),
+	}))
+}
+
+// Sets you in the given games. Specify none to quit all games.
+func (g *GameCoordinator) SetGamesPlayed(appIds ...uint64) {
+	games := make([]*CMsgClientGamesPlayed_GamePlayed, 0)
+	for _, appId := range appIds {
+		games = append(games, &CMsgClientGamesPlayed_GamePlayed{
+			GameId: proto.Uint64(appId),
+		})
+	}
+
+	g.client.Write(NewClientMsgProtobuf(EMsg_ClientGamesPlayed, &CMsgClientGamesPlayed{
+		GamesPlayed: games,
+	}))
+}

vendor/github.com/Philipp15b/go-steam/generator/generator.go

@@ -0,0 +1,295 @@
+/*
+This program generates the protobuf and SteamLanguage files from the SteamKit data.
+*/
+package main
+
+import (
+	"bytes"
+	"fmt"
+	"go/ast"
+	"go/parser"
+	"go/token"
+	"io"
+	"io/ioutil"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"regexp"
+	"runtime"
+	"strings"
+)
+
+var printCommands = false
+
+func main() {
+	args := strings.Join(os.Args[1:], " ")
+
+	found := false
+	if strings.Contains(args, "clean") {
+		clean()
+		found = true
+	}
+	if strings.Contains(args, "steamlang") {
+		buildSteamLanguage()
+		found = true
+	}
+	if strings.Contains(args, "proto") {
+		buildProto()
+		found = true
+	}
+
+	if !found {
+		os.Stderr.WriteString("Invalid target!\nAvailable targets: clean, proto, steamlang\n")
+		os.Exit(1)
+	}
+}
+
+func clean() {
+	print("# Cleaning")
+	cleanGlob("../protocol/**/*.pb.go")
+	cleanGlob("../tf2/protocol/**/*.pb.go")
+	cleanGlob("../dota/protocol/**/*.pb.go")
+
+	os.Remove("../protocol/steamlang/enums.go")
+	os.Remove("../protocol/steamlang/messages.go")
+}
+
+func cleanGlob(pattern string) {
+	protos, _ := filepath.Glob(pattern)
+	for _, proto := range protos {
+		err := os.Remove(proto)
+		if err != nil {
+			panic(err)
+		}
+	}
+}
+
+func buildSteamLanguage() {
+	print("# Building Steam Language")
+	exePath := "./GoSteamLanguageGenerator/bin/Debug/GoSteamLanguageGenerator.exe"
+
+	if runtime.GOOS != "windows" {
+		execute("mono", exePath, "./SteamKit", "../protocol/steamlang")
+	} else {
+		execute(exePath, "./SteamKit", "../protocol/steamlang")
+	}
+	execute("gofmt", "-w", "../protocol/steamlang/enums.go", "../protocol/steamlang/messages.go")
+}
+
+func buildProto() {
+	print("# Building Protobufs")
+
+	buildProtoMap("steamclient", clientProtoFiles, "../protocol/protobuf")
+	buildProtoMap("tf", tf2ProtoFiles, "../tf2/protocol/protobuf")
+	buildProtoMap("dota", dotaProtoFiles, "../dota/protocol/protobuf")
+}
+
+func buildProtoMap(srcSubdir string, files map[string]string, outDir string) {
+	os.MkdirAll(outDir, os.ModePerm)
+	for proto, out := range files {
+		full := filepath.Join(outDir, out)
+		compileProto("SteamKit/Resources/Protobufs", srcSubdir, proto, full)
+		fixProto(full)
+	}
+}
+
+// Maps the proto files to their target files.
+// See `SteamKit/Resources/Protobufs/steamclient/generate-base.bat` for reference.
+var clientProtoFiles = map[string]string{
+	"steammessages_base.proto":   "base.pb.go",
+	"encrypted_app_ticket.proto": "app_ticket.pb.go",
+
+	"steammessages_clientserver.proto":   "client_server.pb.go",
+	"steammessages_clientserver_2.proto": "client_server_2.pb.go",
+
+	"content_manifest.proto": "content_manifest.pb.go",
+
+	"steammessages_unified_base.steamclient.proto":      "unified/base.pb.go",
+	"steammessages_cloud.steamclient.proto":             "unified/cloud.pb.go",
+	"steammessages_credentials.steamclient.proto":       "unified/credentials.pb.go",
+	"steammessages_deviceauth.steamclient.proto":        "unified/deviceauth.pb.go",
+	"steammessages_gamenotifications.steamclient.proto": "unified/gamenotifications.pb.go",
+	"steammessages_offline.steamclient.proto":           "unified/offline.pb.go",
+	"steammessages_parental.steamclient.proto":          "unified/parental.pb.go",
+	"steammessages_partnerapps.steamclient.proto":       "unified/partnerapps.pb.go",
+	"steammessages_player.steamclient.proto":            "unified/player.pb.go",
+	"steammessages_publishedfile.steamclient.proto":     "unified/publishedfile.pb.go",
+}
+
+var tf2ProtoFiles = map[string]string{
+	"base_gcmessages.proto":  "base.pb.go",
+	"econ_gcmessages.proto":  "econ.pb.go",
+	"gcsdk_gcmessages.proto": "gcsdk.pb.go",
+	"tf_gcmessages.proto":    "tf.pb.go",
+	"gcsystemmsgs.proto":     "system.pb.go",
+}
+
+var dotaProtoFiles = map[string]string{
+	"base_gcmessages.proto":                "base.pb.go",
+	"econ_gcmessages.proto":                "econ.pb.go",
+	"gcsdk_gcmessages.proto":               "gcsdk.pb.go",
+	"dota_gcmessages_common.proto":         "dota_common.pb.go",
+	"dota_gcmessages_client.proto":         "dota_client.pb.go",
+	"dota_gcmessages_client_fantasy.proto": "dota_client_fantasy.pb.go",
+	"gcsystemmsgs.proto":                   "system.pb.go",
+}
+
+func compileProto(srcBase, srcSubdir, proto, target string) {
+	outDir, _ := filepath.Split(target)
+	err := os.MkdirAll(outDir, os.ModePerm)
+	if err != nil {
+		panic(err)
+	}
+	execute("protoc", "--go_out="+outDir, "-I="+srcBase+"/"+srcSubdir, "-I="+srcBase, filepath.Join(srcBase, srcSubdir, proto))
+	out := strings.Replace(filepath.Join(outDir, proto), ".proto", ".pb.go", 1)
+	err = forceRename(out, target)
+	if err != nil {
+		panic(err)
+	}
+}
+
+func forceRename(from, to string) error {
+	if from != to {
+		os.Remove(to)
+	}
+	return os.Rename(from, to)
+}
+
+var pkgRegex = regexp.MustCompile(`(package \w+)`)
+var pkgCommentRegex = regexp.MustCompile(`(?s)(\/\*.*?\*\/\n)package`)
+var unusedImportCommentRegex = regexp.MustCompile("// discarding unused import .*\n")
+var fileDescriptorVarRegex = regexp.MustCompile(`fileDescriptor\d+`)
+
+func fixProto(path string) {
+	// goprotobuf is really bad at dependencies, so we must fix them manually...
+	// It tries to load each dependency of a file as a seperate package (but in a very, very wrong way).
+	// Because we want some files in the same package, we'll remove those imports to local files.
+
+	file, err := ioutil.ReadFile(path)
+	if err != nil {
+		panic(err)
+	}
+
+	fset := token.NewFileSet()
+	f, err := parser.ParseFile(fset, path, file, parser.ImportsOnly)
+	if err != nil {
+		panic("Error parsing " + path + ": " + err.Error())
+	}
+
+	importsToRemove := make([]*ast.ImportSpec, 0)
+	for _, i := range f.Imports {
+		// We remove all local imports
+		if i.Path.Value == "\".\"" {
+			importsToRemove = append(importsToRemove, i)
+		}
+	}
+
+	for _, itr := range importsToRemove {
+		// remove the package name from all types
+		file = bytes.Replace(file, []byte(itr.Name.Name+"."), []byte{}, -1)
+		// and remove the import itself
+		file = bytes.Replace(file, []byte(fmt.Sprintf("import %v %v\n", itr.Name.Name, itr.Path.Value)), []byte{}, -1)
+	}
+
+	// remove the package comment because it just includes a list of all messages and
+	// collides not only with the other compiled protobuf files, but also our own documentation.
+	file = cutAllSubmatch(pkgCommentRegex, file, 1)
+
+	// remove warnings
+	file = unusedImportCommentRegex.ReplaceAllLiteral(file, []byte{})
+
+	// fix the package name
+	file = pkgRegex.ReplaceAll(file, []byte("package "+inferPackageName(path)))
+
+	// fix the google dependency;
+	// we just reuse the one from protoc-gen-go
+	file = bytes.Replace(file, []byte("google/protobuf"), []byte("github.com/golang/protobuf/protoc-gen-go/descriptor"), -1)
+
+	// we need to prefix local variables created by protoc-gen-go so that they don't clash with others in the same package
+	filename := strings.Split(filepath.Base(path), ".")[0]
+	file = fileDescriptorVarRegex.ReplaceAllFunc(file, func(match []byte) []byte {
+		return []byte(filename + "_" + string(match))
+	})
+
+	err = ioutil.WriteFile(path, file, os.ModePerm)
+	if err != nil {
+		panic(err)
+	}
+}
+
+func inferPackageName(path string) string {
+	pieces := strings.Split(path, string(filepath.Separator))
+	return pieces[len(pieces)-2]
+}
+
+func cutAllSubmatch(r *regexp.Regexp, b []byte, n int) []byte {
+	i := r.FindSubmatchIndex(b)
+	return bytesCut(b, i[2*n], i[2*n+1])
+}
+
+// Removes the given section from the byte array
+func bytesCut(b []byte, from, to int) []byte {
+	buf := new(bytes.Buffer)
+	buf.Write(b[:from])
+	buf.Write(b[to:])
+	return buf.Bytes()
+}
+
+func print(text string) { os.Stdout.WriteString(text + "\n") }
+
+func printerr(text string) { os.Stderr.WriteString(text + "\n") }
+
+// This writer appends a "> " after every newline so that the outpout appears quoted.
+type QuotedWriter struct {
+	w       io.Writer
+	started bool
+}
+
+func NewQuotedWriter(w io.Writer) *QuotedWriter {
+	return &QuotedWriter{w, false}
+}
+
+func (w *QuotedWriter) Write(p []byte) (n int, err error) {
+	if !w.started {
+		_, err = w.w.Write([]byte("> "))
+		if err != nil {
+			return n, err
+		}
+		w.started = true
+	}
+
+	for i, c := range p {
+		if c == '\n' {
+			nw, err := w.w.Write(p[n : i+1])
+			n += nw
+			if err != nil {
+				return n, err
+			}
+
+			_, err = w.w.Write([]byte("> "))
+			if err != nil {
+				return n, err
+			}
+		}
+	}
+	if n != len(p) {
+		nw, err := w.w.Write(p[n:len(p)])
+		n += nw
+		return n, err
+	}
+	return
+}
+
+func execute(command string, args ...string) {
+	if printCommands {
+		print(command + " " + strings.Join(args, " "))
+	}
+	cmd := exec.Command(command, args...)
+	cmd.Stdout = NewQuotedWriter(os.Stdout)
+	cmd.Stderr = NewQuotedWriter(os.Stderr)
+	err := cmd.Run()
+	if err != nil {
+		printerr(err.Error())
+		os.Exit(1)
+	}
+}

vendor/github.com/Philipp15b/go-steam/gsbot/gsbot.go

@@ -0,0 +1,210 @@
+// The GsBot package contains some useful utilites for working with the
+// steam package. It implements authentication with sentries, server lists and
+// logging messages and events.
+//
+// Every module is optional and requires an instance of the GsBot struct.
+// Should a module have a `HandlePacket` method, you must register it with the
+// steam.Client with `RegisterPacketHandler`. Any module with a `HandleEvent`
+// method must be integrated into your event loop and should be called for each
+// event you receive.
+package gsbot
+
+import (
+	"encoding/hex"
+	"encoding/json"
+	"fmt"
+	"io/ioutil"
+	"log"
+	"math/rand"
+	"net"
+	"os"
+	"path"
+	"reflect"
+	"time"
+
+	"github.com/Philipp15b/go-steam"
+	"github.com/Philipp15b/go-steam/netutil"
+	"github.com/Philipp15b/go-steam/protocol"
+	"github.com/davecgh/go-spew/spew"
+)
+
+// Base structure holding common data among GsBot modules.
+type GsBot struct {
+	Client *steam.Client
+	Log    *log.Logger
+}
+
+// Creates a new GsBot with a new steam.Client where logs are written to stdout.
+func Default() *GsBot {
+	return &GsBot{
+		steam.NewClient(),
+		log.New(os.Stdout, "", 0),
+	}
+}
+
+// This module handles authentication. It logs on automatically after a ConnectedEvent
+// and saves the sentry data to a file which is also used for logon if available.
+// If you're logging on for the first time Steam may require an authcode. You can then
+// connect again with the new logon details.
+type Auth struct {
+	bot             *GsBot
+	details         *LogOnDetails
+	sentryPath      string
+	machineAuthHash []byte
+}
+
+func NewAuth(bot *GsBot, details *LogOnDetails, sentryPath string) *Auth {
+	return &Auth{
+		bot:        bot,
+		details:    details,
+		sentryPath: sentryPath,
+	}
+}
+
+type LogOnDetails struct {
+	Username      string
+	Password      string
+	AuthCode      string
+	TwoFactorCode string
+}
+
+// This is called automatically after every ConnectedEvent, but must be called once again manually
+// with an authcode if Steam requires it when logging on for the first time.
+func (a *Auth) LogOn(details *LogOnDetails) {
+	a.details = details
+	sentry, err := ioutil.ReadFile(a.sentryPath)
+	if err != nil {
+		a.bot.Log.Printf("Error loading sentry file from path %v - This is normal if you're logging in for the first time.\n", a.sentryPath)
+	}
+	a.bot.Client.Auth.LogOn(&steam.LogOnDetails{
+		Username:       details.Username,
+		Password:       details.Password,
+		SentryFileHash: sentry,
+		AuthCode:       details.AuthCode,
+		TwoFactorCode:  details.TwoFactorCode,
+	})
+}
+
+func (a *Auth) HandleEvent(event interface{}) {
+	switch e := event.(type) {
+	case *steam.ConnectedEvent:
+		a.LogOn(a.details)
+	case *steam.LoggedOnEvent:
+		a.bot.Log.Printf("Logged on (%v) with SteamId %v and account flags %v", e.Result, e.ClientSteamId, e.AccountFlags)
+	case *steam.MachineAuthUpdateEvent:
+		a.machineAuthHash = e.Hash
+		err := ioutil.WriteFile(a.sentryPath, e.Hash, 0666)
+		if err != nil {
+			panic(err)
+		}
+	}
+}
+
+// This module saves the server list from ClientCMListEvent and uses
+// it when you call `Connect()`.
+type ServerList struct {
+	bot      *GsBot
+	listPath string
+}
+
+func NewServerList(bot *GsBot, listPath string) *ServerList {
+	return &ServerList{
+		bot,
+		listPath,
+	}
+}
+
+func (s *ServerList) HandleEvent(event interface{}) {
+	switch e := event.(type) {
+	case *steam.ClientCMListEvent:
+		d, err := json.Marshal(e.Addresses)
+		if err != nil {
+			panic(err)
+		}
+		err = ioutil.WriteFile(s.listPath, d, 0666)
+		if err != nil {
+			panic(err)
+		}
+	}
+}
+
+func (s *ServerList) Connect() (bool, error) {
+	return s.ConnectBind(nil)
+}
+
+func (s *ServerList) ConnectBind(laddr *net.TCPAddr) (bool, error) {
+	d, err := ioutil.ReadFile(s.listPath)
+	if err != nil {
+		s.bot.Log.Println("Connecting to random server.")
+		s.bot.Client.Connect()
+		return false, nil
+	}
+	var addrs []*netutil.PortAddr
+	err = json.Unmarshal(d, &addrs)
+	if err != nil {
+		return false, err
+	}
+	raddr := addrs[rand.Intn(len(addrs))]
+	s.bot.Log.Printf("Connecting to %v from server list\n", raddr)
+	s.bot.Client.ConnectToBind(raddr, laddr)
+	return true, nil
+}
+
+// This module logs incoming packets and events to a directory.
+type Debug struct {
+	packetId, eventId uint64
+	bot               *GsBot
+	base              string
+}
+
+func NewDebug(bot *GsBot, base string) (*Debug, error) {
+	base = path.Join(base, fmt.Sprint(time.Now().Unix()))
+	err := os.MkdirAll(path.Join(base, "events"), 0700)
+	if err != nil {
+		return nil, err
+	}
+	err = os.MkdirAll(path.Join(base, "packets"), 0700)
+	if err != nil {
+		return nil, err
+	}
+	return &Debug{
+		0, 0,
+		bot,
+		base,
+	}, nil
+}
+
+func (d *Debug) HandlePacket(packet *protocol.Packet) {
+	d.packetId++
+	name := path.Join(d.base, "packets", fmt.Sprintf("%d_%d_%s", time.Now().Unix(), d.packetId, packet.EMsg))
+
+	text := packet.String() + "\n\n" + hex.Dump(packet.Data)
+	err := ioutil.WriteFile(name+".txt", []byte(text), 0666)
+	if err != nil {
+		panic(err)
+	}
+
+	err = ioutil.WriteFile(name+".bin", packet.Data, 0666)
+	if err != nil {
+		panic(err)
+	}
+}
+
+func (d *Debug) HandleEvent(event interface{}) {
+	d.eventId++
+	name := fmt.Sprintf("%d_%d_%s.txt", time.Now().Unix(), d.eventId, name(event))
+	err := ioutil.WriteFile(path.Join(d.base, "events", name), []byte(spew.Sdump(event)), 0666)
+	if err != nil {
+		panic(err)
+	}
+}
+
+func name(obj interface{}) string {
+	val := reflect.ValueOf(obj)
+	ind := reflect.Indirect(val)
+	if ind.IsValid() {
+		return ind.Type().Name()
+	} else {
+		return val.Type().Name()
+	}
+}

vendor/github.com/Philipp15b/go-steam/gsbot/gsbot/gsbot.go

@@ -0,0 +1,56 @@
+// A simple example that uses the modules from the gsbot package and go-steam to log on
+// to the Steam network.
+//
+// The command expects log on data, optionally with an auth code:
+//
+//     gsbot [username] [password]
+//     gsbot [username] [password] [authcode]
+package main
+
+import (
+	"fmt"
+	"os"
+
+	"github.com/Philipp15b/go-steam"
+	"github.com/Philipp15b/go-steam/gsbot"
+	"github.com/Philipp15b/go-steam/protocol/steamlang"
+)
+
+func main() {
+	if len(os.Args) < 3 {
+		fmt.Println("gsbot example\nusage: \n\tgsbot [username] [password] [authcode]")
+		return
+	}
+	authcode := ""
+	if len(os.Args) > 3 {
+		authcode = os.Args[3]
+	}
+
+	bot := gsbot.Default()
+	client := bot.Client
+	auth := gsbot.NewAuth(bot, &gsbot.LogOnDetails{
+		os.Args[1],
+		os.Args[2],
+		authcode,
+	}, "sentry.bin")
+	debug, err := gsbot.NewDebug(bot, "debug")
+	if err != nil {
+		panic(err)
+	}
+	client.RegisterPacketHandler(debug)
+	serverList := gsbot.NewServerList(bot, "serverlist.json")
+	serverList.Connect()
+
+	for event := range client.Events() {
+		auth.HandleEvent(event)
+		debug.HandleEvent(event)
+		serverList.HandleEvent(event)
+
+		switch e := event.(type) {
+		case error:
+			fmt.Printf("Error: %v", e)
+		case *steam.LoggedOnEvent:
+			client.Social.SetPersonaState(steamlang.EPersonaState_Online)
+		}
+	}
+}

vendor/github.com/Philipp15b/go-steam/jsont/jsont.go

@@ -0,0 +1,19 @@
+// Includes helper types for working with JSON data
+package jsont
+
+import (
+	"encoding/json"
+)
+
+// A boolean value that can be unmarshaled from a number in JSON.
+type UintBool bool
+
+func (u *UintBool) UnmarshalJSON(data []byte) error {
+	var n uint
+	err := json.Unmarshal(data, &n)
+	if err != nil {
+		return err
+	}
+	*u = n != 0
+	return nil
+}

vendor/github.com/Philipp15b/go-steam/keys.go

@@ -0,0 +1,58 @@
+package steam
+
+import (
+	"crypto/rsa"
+	"github.com/Philipp15b/go-steam/cryptoutil"
+	. "github.com/Philipp15b/go-steam/protocol/steamlang"
+)
+
+var publicKeys = map[EUniverse][]byte{
+	EUniverse_Public: []byte{
+		0x30, 0x81, 0x9D, 0x30, 0x0D, 0x06, 0x09, 0x2A, 0x86, 0x48, 0x86, 0xF7, 0x0D, 0x01, 0x01, 0x01,
+		0x05, 0x00, 0x03, 0x81, 0x8B, 0x00, 0x30, 0x81, 0x87, 0x02, 0x81, 0x81, 0x00, 0xDF, 0xEC, 0x1A,
+		0xD6, 0x2C, 0x10, 0x66, 0x2C, 0x17, 0x35, 0x3A, 0x14, 0xB0, 0x7C, 0x59, 0x11, 0x7F, 0x9D, 0xD3,
+		0xD8, 0x2B, 0x7A, 0xE3, 0xE0, 0x15, 0xCD, 0x19, 0x1E, 0x46, 0xE8, 0x7B, 0x87, 0x74, 0xA2, 0x18,
+		0x46, 0x31, 0xA9, 0x03, 0x14, 0x79, 0x82, 0x8E, 0xE9, 0x45, 0xA2, 0x49, 0x12, 0xA9, 0x23, 0x68,
+		0x73, 0x89, 0xCF, 0x69, 0xA1, 0xB1, 0x61, 0x46, 0xBD, 0xC1, 0xBE, 0xBF, 0xD6, 0x01, 0x1B, 0xD8,
+		0x81, 0xD4, 0xDC, 0x90, 0xFB, 0xFE, 0x4F, 0x52, 0x73, 0x66, 0xCB, 0x95, 0x70, 0xD7, 0xC5, 0x8E,
+		0xBA, 0x1C, 0x7A, 0x33, 0x75, 0xA1, 0x62, 0x34, 0x46, 0xBB, 0x60, 0xB7, 0x80, 0x68, 0xFA, 0x13,
+		0xA7, 0x7A, 0x8A, 0x37, 0x4B, 0x9E, 0xC6, 0xF4, 0x5D, 0x5F, 0x3A, 0x99, 0xF9, 0x9E, 0xC4, 0x3A,
+		0xE9, 0x63, 0xA2, 0xBB, 0x88, 0x19, 0x28, 0xE0, 0xE7, 0x14, 0xC0, 0x42, 0x89, 0x02, 0x01, 0x11,
+	},
+
+	EUniverse_Beta: []byte{
+		0x30, 0x81, 0x9D, 0x30, 0x0D, 0x06, 0x09, 0x2A, 0x86, 0x48, 0x86, 0xF7, 0x0D, 0x01, 0x01, 0x01,
+		0x05, 0x00, 0x03, 0x81, 0x8B, 0x00, 0x30, 0x81, 0x87, 0x02, 0x81, 0x81, 0x00, 0xAE, 0xD1, 0x4B,
+		0xC0, 0xA3, 0x36, 0x8B, 0xA0, 0x39, 0x0B, 0x43, 0xDC, 0xED, 0x6A, 0xC8, 0xF2, 0xA3, 0xE4, 0x7E,
+		0x09, 0x8C, 0x55, 0x2E, 0xE7, 0xE9, 0x3C, 0xBB, 0xE5, 0x5E, 0x0F, 0x18, 0x74, 0x54, 0x8F, 0xF3,
+		0xBD, 0x56, 0x69, 0x5B, 0x13, 0x09, 0xAF, 0xC8, 0xBE, 0xB3, 0xA1, 0x48, 0x69, 0xE9, 0x83, 0x49,
+		0x65, 0x8D, 0xD2, 0x93, 0x21, 0x2F, 0xB9, 0x1E, 0xFA, 0x74, 0x3B, 0x55, 0x22, 0x79, 0xBF, 0x85,
+		0x18, 0xCB, 0x6D, 0x52, 0x44, 0x4E, 0x05, 0x92, 0x89, 0x6A, 0xA8, 0x99, 0xED, 0x44, 0xAE, 0xE2,
+		0x66, 0x46, 0x42, 0x0C, 0xFB, 0x6E, 0x4C, 0x30, 0xC6, 0x6C, 0x5C, 0x16, 0xFF, 0xBA, 0x9C, 0xB9,
+		0x78, 0x3F, 0x17, 0x4B, 0xCB, 0xC9, 0x01, 0x5D, 0x3E, 0x37, 0x70, 0xEC, 0x67, 0x5A, 0x33, 0x48,
+	},
+
+	EUniverse_Internal: []byte{
+		0x30, 0x81, 0x9D, 0x30, 0x0D, 0x06, 0x09, 0x2A, 0x86, 0x48, 0x86, 0xF7, 0x0D, 0x01, 0x01, 0x01,
+		0x05, 0x00, 0x03, 0x81, 0x8B, 0x00, 0x30, 0x81, 0x87, 0x02, 0x81, 0x81, 0x00, 0xA8, 0xFE, 0x01,
+		0x3B, 0xB6, 0xD7, 0x21, 0x4B, 0x53, 0x23, 0x6F, 0xA1, 0xAB, 0x4E, 0xF1, 0x07, 0x30, 0xA7, 0xC6,
+		0x7E, 0x6A, 0x2C, 0xC2, 0x5D, 0x3A, 0xB8, 0x40, 0xCA, 0x59, 0x4D, 0x16, 0x2D, 0x74, 0xEB, 0x0E,
+		0x72, 0x46, 0x29, 0xF9, 0xDE, 0x9B, 0xCE, 0x4B, 0x8C, 0xD0, 0xCA, 0xF4, 0x08, 0x94, 0x46, 0xA5,
+		0x11, 0xAF, 0x3A, 0xCB, 0xB8, 0x4E, 0xDE, 0xC6, 0xD8, 0x85, 0x0A, 0x7D, 0xAA, 0x96, 0x0A, 0xEA,
+		0x7B, 0x51, 0xD6, 0x22, 0x62, 0x5C, 0x1E, 0x58, 0xD7, 0x46, 0x1E, 0x09, 0xAE, 0x43, 0xA7, 0xC4,
+		0x34, 0x69, 0xA2, 0xA5, 0xE8, 0x44, 0x76, 0x18, 0xE2, 0x3D, 0xB7, 0xC5, 0xA8, 0x96, 0xFD, 0xE5,
+		0xB4, 0x4B, 0xF8, 0x40, 0x12, 0xA6, 0x17, 0x4E, 0xC4, 0xC1, 0x60, 0x0E, 0xB0, 0xC2, 0xB8, 0x40,
+	},
+}
+
+func GetPublicKey(universe EUniverse) *rsa.PublicKey {
+	bytes, ok := publicKeys[universe]
+	if !ok {
+		return nil
+	}
+	key, err := cryptoutil.ParseASN1RSAPublicKey(bytes)
+	if err != nil {
+		panic(err)
+	}
+	return key
+}

vendor/github.com/Philipp15b/go-steam/netutil/addr.go

@@ -0,0 +1,43 @@
+package netutil
+
+import (
+	"net"
+	"strconv"
+	"strings"
+)
+
+// An addr that is neither restricted to TCP nor UDP, but has an IP and a port.
+type PortAddr struct {
+	IP   net.IP
+	Port uint16
+}
+
+// Parses an IP address with a port, for example "209.197.29.196:27017".
+// If the given string is not valid, this function returns nil.
+func ParsePortAddr(addr string) *PortAddr {
+	parts := strings.Split(addr, ":")
+	if len(parts) != 2 {
+		return nil
+	}
+	ip := net.ParseIP(parts[0])
+	if ip == nil {
+		return nil
+	}
+	port, err := strconv.ParseUint(parts[1], 10, 16)
+	if err != nil {
+		return nil
+	}
+	return &PortAddr{ip, uint16(port)}
+}
+
+func (p *PortAddr) ToTCPAddr() *net.TCPAddr {
+	return &net.TCPAddr{p.IP, int(p.Port), ""}
+}
+
+func (p *PortAddr) ToUDPAddr() *net.UDPAddr {
+	return &net.UDPAddr{p.IP, int(p.Port), ""}
+}
+
+func (p *PortAddr) String() string {
+	return p.IP.String() + ":" + strconv.FormatUint(uint64(p.Port), 10)
+}

vendor/github.com/Philipp15b/go-steam/netutil/http.go

@@ -0,0 +1,17 @@
+package netutil
+
+import (
+	"net/http"
+	"net/url"
+	"strings"
+)
+
+// Version of http.Client.PostForm that returns a new request instead of executing it directly.
+func NewPostForm(url string, data url.Values) *http.Request {
+	req, err := http.NewRequest("POST", url, strings.NewReader(data.Encode()))
+	if err != nil {
+		panic(err)
+	}
+	req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
+	return req
+}

vendor/github.com/Philipp15b/go-steam/netutil/url.go

@@ -0,0 +1,13 @@
+package netutil
+
+import (
+	"net/url"
+)
+
+func ToUrlValues(m map[string]string) url.Values {
+	r := make(url.Values)
+	for k, v := range m {
+		r.Add(k, v)
+	}
+	return r
+}

vendor/github.com/Philipp15b/go-steam/notifications.go

@@ -0,0 +1,62 @@
+package steam
+
+import (
+	. "github.com/Philipp15b/go-steam/protocol"
+	. "github.com/Philipp15b/go-steam/protocol/protobuf"
+	. "github.com/Philipp15b/go-steam/protocol/steamlang"
+)
+
+type Notifications struct {
+	// Maps notification types to their count. If a type is not present in the map,
+	// its count is zero.
+	notifications map[NotificationType]uint
+	client        *Client
+}
+
+func newNotifications(client *Client) *Notifications {
+	return &Notifications{
+		make(map[NotificationType]uint),
+		client,
+	}
+}
+
+func (n *Notifications) HandlePacket(packet *Packet) {
+	switch packet.EMsg {
+	case EMsg_ClientUserNotifications:
+		n.handleClientUserNotifications(packet)
+	}
+}
+
+type NotificationType uint
+
+const (
+	TradeOffer NotificationType = 1
+)
+
+func (n *Notifications) handleClientUserNotifications(packet *Packet) {
+	msg := new(CMsgClientUserNotifications)
+	packet.ReadProtoMsg(msg)
+
+	for _, notification := range msg.GetNotifications() {
+		typ := NotificationType(*notification.UserNotificationType)
+		count := uint(*notification.Count)
+		n.notifications[typ] = count
+		n.client.Emit(&NotificationEvent{typ, count})
+	}
+
+	// check if there is a notification in our map that isn't in the current packet
+	for typ, _ := range n.notifications {
+		exists := false
+		for _, t := range msg.GetNotifications() {
+			if NotificationType(*t.UserNotificationType) == typ {
+				exists = true
+				break
+			}
+		}
+
+		if !exists {
+			delete(n.notifications, typ)
+			n.client.Emit(&NotificationEvent{typ, 0})
+		}
+	}
+}

vendor/github.com/Philipp15b/go-steam/notifications_events.go

@@ -0,0 +1,9 @@
+package steam
+
+// This event is emitted for every CMsgClientUserNotifications message and likewise only used for
+// trade offers. Unlike the the above it is also emitted when the count of a type that was tracked
+// before by this Notifications instance reaches zero.
+type NotificationEvent struct {
+	Type  NotificationType
+	Count uint
+}

vendor/github.com/Philipp15b/go-steam/protocol/doc.go

@@ -0,0 +1,18 @@
+/*
+This package includes some basics for the Steam protocol. It defines basic interfaces that are used throughout go-steam:
+There is IMsg, which is extended by IClientMsg (sent after logging in) and abstracts over
+the outgoing message types. Both interfaces are implemented by ClientMsgProtobuf and ClientMsg.
+Msg is like ClientMsg, but it is used for sending messages before logging in.
+
+There is also the concept of a Packet: This is a type for incoming messages where only
+the header is deserialized. It therefore only contains EMsg data, job information and the remaining data.
+Its contents can then be read via the Read* methods which read data into a MessageBody - a type which is Serializable and
+has an EMsg.
+
+In addition, there are extra types for communication with the Game Coordinator (GC) included in the gamecoordinator sub-package.
+For outgoing messages the IGCMsg interface is used which is implemented by GCMsgProtobuf and GCMsg.
+Incoming messages are of the GCPacket type and are read like regular Packets.
+
+The actual messages and enums are in the sub-packages steamlang and protobuf, generated from the SteamKit data.
+*/
+package protocol

vendor/github.com/Philipp15b/go-steam/protocol/gamecoordinator/msg.go

@@ -0,0 +1,132 @@
+package gamecoordinator
+
+import (
+	"io"
+
+	. "github.com/Philipp15b/go-steam/protocol"
+	. "github.com/Philipp15b/go-steam/protocol/steamlang"
+	"github.com/golang/protobuf/proto"
+)
+
+// An outgoing message to the Game Coordinator.
+type IGCMsg interface {
+	Serializer
+	IsProto() bool
+	GetAppId() uint32
+	GetMsgType() uint32
+
+	GetTargetJobId() JobId
+	SetTargetJobId(JobId)
+	GetSourceJobId() JobId
+	SetSourceJobId(JobId)
+}
+
+type GCMsgProtobuf struct {
+	AppId  uint32
+	Header *MsgGCHdrProtoBuf
+	Body   proto.Message
+}
+
+func NewGCMsgProtobuf(appId, msgType uint32, body proto.Message) *GCMsgProtobuf {
+	hdr := NewMsgGCHdrProtoBuf()
+	hdr.Msg = msgType
+	return &GCMsgProtobuf{
+		AppId:  appId,
+		Header: hdr,
+		Body:   body,
+	}
+}
+
+func (g *GCMsgProtobuf) IsProto() bool {
+	return true
+}
+
+func (g *GCMsgProtobuf) GetAppId() uint32 {
+	return g.AppId
+}
+
+func (g *GCMsgProtobuf) GetMsgType() uint32 {
+	return g.Header.Msg
+}
+
+func (g *GCMsgProtobuf) GetTargetJobId() JobId {
+	return JobId(g.Header.Proto.GetJobidTarget())
+}
+
+func (g *GCMsgProtobuf) SetTargetJobId(job JobId) {
+	g.Header.Proto.JobidTarget = proto.Uint64(uint64(job))
+}
+
+func (g *GCMsgProtobuf) GetSourceJobId() JobId {
+	return JobId(g.Header.Proto.GetJobidSource())
+}
+
+func (g *GCMsgProtobuf) SetSourceJobId(job JobId) {
+	g.Header.Proto.JobidSource = proto.Uint64(uint64(job))
+}
+
+func (g *GCMsgProtobuf) Serialize(w io.Writer) error {
+	err := g.Header.Serialize(w)
+	if err != nil {
+		return err
+	}
+	body, err := proto.Marshal(g.Body)
+	if err != nil {
+		return err
+	}
+	_, err = w.Write(body)
+	return err
+}
+
+type GCMsg struct {
+	AppId   uint32
+	MsgType uint32
+	Header  *MsgGCHdr
+	Body    Serializer
+}
+
+func NewGCMsg(appId, msgType uint32, body Serializer) *GCMsg {
+	return &GCMsg{
+		AppId:   appId,
+		MsgType: msgType,
+		Header:  NewMsgGCHdr(),
+		Body:    body,
+	}
+}
+
+func (g *GCMsg) GetMsgType() uint32 {
+	return g.MsgType
+}
+
+func (g *GCMsg) GetAppId() uint32 {
+	return g.AppId
+}
+
+func (g *GCMsg) IsProto() bool {
+	return false
+}
+
+func (g *GCMsg) GetTargetJobId() JobId {
+	return JobId(g.Header.TargetJobID)
+}
+
+func (g *GCMsg) SetTargetJobId(job JobId) {
+	g.Header.TargetJobID = uint64(job)
+}
+
+func (g *GCMsg) GetSourceJobId() JobId {
+	return JobId(g.Header.SourceJobID)
+}
+
+func (g *GCMsg) SetSourceJobId(job JobId) {
+	g.Header.SourceJobID = uint64(job)
+}
+
+func (g *GCMsg) Serialize(w io.Writer) error {
+	err := g.Header.Serialize(w)
+	if err != nil {
+		return err
+	}
+	err = g.Body.Serialize(w)
+	return err
+}

vendor/github.com/Philipp15b/go-steam/protocol/gamecoordinator/packet.go

@@ -0,0 +1,61 @@
+package gamecoordinator
+
+import (
+	"bytes"
+	. "github.com/Philipp15b/go-steam/protocol"
+	. "github.com/Philipp15b/go-steam/protocol/protobuf"
+	. "github.com/Philipp15b/go-steam/protocol/steamlang"
+	"github.com/golang/protobuf/proto"
+)
+
+// An incoming, partially unread message from the Game Coordinator.
+type GCPacket struct {
+	AppId       uint32
+	MsgType     uint32
+	IsProto     bool
+	GCName      string
+	Body        []byte
+	TargetJobId JobId
+}
+
+func NewGCPacket(wrapper *CMsgGCClient) (*GCPacket, error) {
+	packet := &GCPacket{
+		AppId:   wrapper.GetAppid(),
+		MsgType: wrapper.GetMsgtype(),
+		GCName:  wrapper.GetGcname(),
+	}
+
+	r := bytes.NewReader(wrapper.GetPayload())
+	if IsProto(wrapper.GetMsgtype()) {
+		packet.MsgType = packet.MsgType & EMsgMask
+		packet.IsProto = true
+
+		header := NewMsgGCHdrProtoBuf()
+		err := header.Deserialize(r)
+		if err != nil {
+			return nil, err
+		}
+		packet.TargetJobId = JobId(header.Proto.GetJobidTarget())
+	} else {
+		header := NewMsgGCHdr()
+		err := header.Deserialize(r)
+		if err != nil {
+			return nil, err
+		}
+		packet.TargetJobId = JobId(header.TargetJobID)
+	}
+
+	body := make([]byte, r.Len())
+	r.Read(body)
+	packet.Body = body
+
+	return packet, nil
+}
+
+func (g *GCPacket) ReadProtoMsg(body proto.Message) {
+	proto.Unmarshal(g.Body, body)
+}
+
+func (g *GCPacket) ReadMsg(body MessageBody) {
+	body.Deserialize(bytes.NewReader(g.Body))
+}

vendor/github.com/Philipp15b/go-steam/protocol/internal.go

@@ -0,0 +1,47 @@
+package protocol
+
+import (
+	"io"
+	"math"
+	"strconv"
+
+	. "github.com/Philipp15b/go-steam/protocol/steamlang"
+)
+
+type JobId uint64
+
+func (j JobId) String() string {
+	if j == math.MaxUint64 {
+		return "(none)"
+	}
+	return strconv.FormatUint(uint64(j), 10)
+}
+
+type Serializer interface {
+	Serialize(w io.Writer) error
+}
+
+type Deserializer interface {
+	Deserialize(r io.Reader) error
+}
+
+type Serializable interface {
+	Serializer
+	Deserializer
+}
+
+type MessageBody interface {
+	Serializable
+	GetEMsg() EMsg
+}
+
+// the default details to request in most situations
+const EClientPersonaStateFlag_DefaultInfoRequest = EClientPersonaStateFlag_PlayerName |
+	EClientPersonaStateFlag_Presence | EClientPersonaStateFlag_SourceID |
+	EClientPersonaStateFlag_GameExtraInfo
+
+const DefaultAvatar = "fef49e7fa7e1997310d705b2a6158ff8dc1cdfeb"
+
+func ValidAvatar(avatar string) bool {
+	return !(avatar == "0000000000000000000000000000000000000000" || len(avatar) != 40)
+}

vendor/github.com/Philipp15b/go-steam/protocol/msg.go

@@ -0,0 +1,221 @@
+package protocol
+
+import (
+	"github.com/golang/protobuf/proto"
+	. "github.com/Philipp15b/go-steam/protocol/steamlang"
+	. "github.com/Philipp15b/go-steam/steamid"
+	"io"
+)
+
+// Interface for all messages, typically outgoing. They can also be created by
+// using the Read* methods in a PacketMsg.
+type IMsg interface {
+	Serializer
+	IsProto() bool
+	GetMsgType() EMsg
+	GetTargetJobId() JobId
+	SetTargetJobId(JobId)
+	GetSourceJobId() JobId
+	SetSourceJobId(JobId)
+}
+
+// Interface for client messages, i.e. messages that are sent after logging in.
+// ClientMsgProtobuf and ClientMsg implement this.
+type IClientMsg interface {
+	IMsg
+	GetSessionId() int32
+	SetSessionId(int32)
+	GetSteamId() SteamId
+	SetSteamId(SteamId)
+}
+
+// Represents a protobuf backed client message with session data.
+type ClientMsgProtobuf struct {
+	Header *MsgHdrProtoBuf
+	Body   proto.Message
+}
+
+func NewClientMsgProtobuf(eMsg EMsg, body proto.Message) *ClientMsgProtobuf {
+	hdr := NewMsgHdrProtoBuf()
+	hdr.Msg = eMsg
+	return &ClientMsgProtobuf{
+		Header: hdr,
+		Body:   body,
+	}
+}
+
+func (c *ClientMsgProtobuf) IsProto() bool {
+	return true
+}
+
+func (c *ClientMsgProtobuf) GetMsgType() EMsg {
+	return NewEMsg(uint32(c.Header.Msg))
+}
+
+func (c *ClientMsgProtobuf) GetSessionId() int32 {
+	return c.Header.Proto.GetClientSessionid()
+}
+
+func (c *ClientMsgProtobuf) SetSessionId(session int32) {
+	c.Header.Proto.ClientSessionid = &session
+}
+
+func (c *ClientMsgProtobuf) GetSteamId() SteamId {
+	return SteamId(c.Header.Proto.GetSteamid())
+}
+
+func (c *ClientMsgProtobuf) SetSteamId(s SteamId) {
+	c.Header.Proto.Steamid = proto.Uint64(uint64(s))
+}
+
+func (c *ClientMsgProtobuf) GetTargetJobId() JobId {
+	return JobId(c.Header.Proto.GetJobidTarget())
+}
+
+func (c *ClientMsgProtobuf) SetTargetJobId(job JobId) {
+	c.Header.Proto.JobidTarget = proto.Uint64(uint64(job))
+}
+
+func (c *ClientMsgProtobuf) GetSourceJobId() JobId {
+	return JobId(c.Header.Proto.GetJobidSource())
+}
+
+func (c *ClientMsgProtobuf) SetSourceJobId(job JobId) {
+	c.Header.Proto.JobidSource = proto.Uint64(uint64(job))
+}
+
+func (c *ClientMsgProtobuf) Serialize(w io.Writer) error {
+	err := c.Header.Serialize(w)
+	if err != nil {
+		return err
+	}
+	body, err := proto.Marshal(c.Body)
+	if err != nil {
+		return err
+	}
+	_, err = w.Write(body)
+	return err
+}
+
+// Represents a struct backed client message.
+type ClientMsg struct {
+	Header  *ExtendedClientMsgHdr
+	Body    MessageBody
+	Payload []byte
+}
+
+func NewClientMsg(body MessageBody, payload []byte) *ClientMsg {
+	hdr := NewExtendedClientMsgHdr()
+	hdr.Msg = body.GetEMsg()
+	return &ClientMsg{
+		Header:  hdr,
+		Body:    body,
+		Payload: payload,
+	}
+}
+
+func (c *ClientMsg) IsProto() bool {
+	return true
+}
+
+func (c *ClientMsg) GetMsgType() EMsg {
+	return c.Header.Msg
+}
+
+func (c *ClientMsg) GetSessionId() int32 {
+	return c.Header.SessionID
+}
+
+func (c *ClientMsg) SetSessionId(session int32) {
+	c.Header.SessionID = session
+}
+
+func (c *ClientMsg) GetSteamId() SteamId {
+	return c.Header.SteamID
+}
+
+func (c *ClientMsg) SetSteamId(s SteamId) {
+	c.Header.SteamID = s
+}
+
+func (c *ClientMsg) GetTargetJobId() JobId {
+	return JobId(c.Header.TargetJobID)
+}
+
+func (c *ClientMsg) SetTargetJobId(job JobId) {
+	c.Header.TargetJobID = uint64(job)
+}
+
+func (c *ClientMsg) GetSourceJobId() JobId {
+	return JobId(c.Header.SourceJobID)
+}
+
+func (c *ClientMsg) SetSourceJobId(job JobId) {
+	c.Header.SourceJobID = uint64(job)
+}
+
+func (c *ClientMsg) Serialize(w io.Writer) error {
+	err := c.Header.Serialize(w)
+	if err != nil {
+		return err
+	}
+	err = c.Body.Serialize(w)
+	if err != nil {
+		return err
+	}
+	_, err = w.Write(c.Payload)
+	return err
+}
+
+type Msg struct {
+	Header  *MsgHdr
+	Body    MessageBody
+	Payload []byte
+}
+
+func NewMsg(body MessageBody, payload []byte) *Msg {
+	hdr := NewMsgHdr()
+	hdr.Msg = body.GetEMsg()
+	return &Msg{
+		Header:  hdr,
+		Body:    body,
+		Payload: payload,
+	}
+}
+
+func (m *Msg) GetMsgType() EMsg {
+	return m.Header.Msg
+}
+
+func (m *Msg) IsProto() bool {
+	return false
+}
+
+func (m *Msg) GetTargetJobId() JobId {
+	return JobId(m.Header.TargetJobID)
+}
+
+func (m *Msg) SetTargetJobId(job JobId) {
+	m.Header.TargetJobID = uint64(job)
+}
+
+func (m *Msg) GetSourceJobId() JobId {
+	return JobId(m.Header.SourceJobID)
+}
+
+func (m *Msg) SetSourceJobId(job JobId) {
+	m.Header.SourceJobID = uint64(job)
+}
+
+func (m *Msg) Serialize(w io.Writer) error {
+	err := m.Header.Serialize(w)
+	if err != nil {
+		return err
+	}
+	err = m.Body.Serialize(w)
+	if err != nil {
+		return err
+	}
+	_, err = w.Write(m.Payload)
+	return err
+}

vendor/github.com/Philipp15b/go-steam/protocol/packet.go

@@ -0,0 +1,116 @@
+package protocol
+
+import (
+	"bytes"
+	"github.com/golang/protobuf/proto"
+	"encoding/binary"
+	"fmt"
+	. "github.com/Philipp15b/go-steam/protocol/steamlang"
+)
+
+// TODO: Headers are always deserialized twice.
+
+// Represents an incoming, partially unread message.
+type Packet struct {
+	EMsg        EMsg
+	IsProto     bool
+	TargetJobId JobId
+	SourceJobId JobId
+	Data        []byte
+}
+
+func NewPacket(data []byte) (*Packet, error) {
+	var rawEMsg uint32
+	err := binary.Read(bytes.NewReader(data), binary.LittleEndian, &rawEMsg)
+	if err != nil {
+		return nil, err
+	}
+	eMsg := NewEMsg(rawEMsg)
+	buf := bytes.NewReader(data)
+	if eMsg == EMsg_ChannelEncryptRequest || eMsg == EMsg_ChannelEncryptResult {
+		header := NewMsgHdr()
+		header.Msg = eMsg
+		err = header.Deserialize(buf)
+		if err != nil {
+			return nil, err
+		}
+		return &Packet{
+			EMsg:        eMsg,
+			IsProto:     false,
+			TargetJobId: JobId(header.TargetJobID),
+			SourceJobId: JobId(header.SourceJobID),
+			Data:        data,
+		}, nil
+	} else if IsProto(rawEMsg) {
+		header := NewMsgHdrProtoBuf()
+		header.Msg = eMsg
+		err = header.Deserialize(buf)
+		if err != nil {
+			return nil, err
+		}
+		return &Packet{
+			EMsg:        eMsg,
+			IsProto:     true,
+			TargetJobId: JobId(header.Proto.GetJobidTarget()),
+			SourceJobId: JobId(header.Proto.GetJobidSource()),
+			Data:        data,
+		}, nil
+	} else {
+		header := NewExtendedClientMsgHdr()
+		header.Msg = eMsg
+		err = header.Deserialize(buf)
+		if err != nil {
+			return nil, err
+		}
+		return &Packet{
+			EMsg:        eMsg,
+			IsProto:     false,
+			TargetJobId: JobId(header.TargetJobID),
+			SourceJobId: JobId(header.SourceJobID),
+			Data:        data,
+		}, nil
+	}
+}
+
+func (p *Packet) String() string {
+	return fmt.Sprintf("Packet{EMsg = %v, Proto = %v, Len = %v, TargetJobId = %v, SourceJobId = %v}", p.EMsg, p.IsProto, len(p.Data), p.TargetJobId, p.SourceJobId)
+}
+
+func (p *Packet) ReadProtoMsg(body proto.Message) *ClientMsgProtobuf {
+	header := NewMsgHdrProtoBuf()
+	buf := bytes.NewBuffer(p.Data)
+	header.Deserialize(buf)
+	proto.Unmarshal(buf.Bytes(), body)
+	return &ClientMsgProtobuf{ // protobuf messages have no payload
+		Header: header,
+		Body:   body,
+	}
+}
+
+func (p *Packet) ReadClientMsg(body MessageBody) *ClientMsg {
+	header := NewExtendedClientMsgHdr()
+	buf := bytes.NewReader(p.Data)
+	header.Deserialize(buf)
+	body.Deserialize(buf)
+	payload := make([]byte, buf.Len())
+	buf.Read(payload)
+	return &ClientMsg{
+		Header:  header,
+		Body:    body,
+		Payload: payload,
+	}
+}
+
+func (p *Packet) ReadMsg(body MessageBody) *Msg {
+	header := NewMsgHdr()
+	buf := bytes.NewReader(p.Data)
+	header.Deserialize(buf)
+	body.Deserialize(buf)
+	payload := make([]byte, buf.Len())
+	buf.Read(payload)
+	return &Msg{
+		Header:  header,
+		Body:    body,
+		Payload: payload,
+	}
+}

vendor/github.com/Philipp15b/go-steam/protocol/protobuf/app_ticket.pb.go

@@ -0,0 +1,82 @@
+// Code generated by protoc-gen-go.
+// source: encrypted_app_ticket.proto
+// DO NOT EDIT!
+
+package protobuf
+
+import proto "github.com/golang/protobuf/proto"
+import fmt "fmt"
+import math "math"
+
+// Reference imports to suppress errors if they are not otherwise used.
+var _ = proto.Marshal
+var _ = fmt.Errorf
+var _ = math.Inf
+
+type EncryptedAppTicket struct {
+	TicketVersionNo               *uint32 `protobuf:"varint,1,opt,name=ticket_version_no" json:"ticket_version_no,omitempty"`
+	CrcEncryptedticket            *uint32 `protobuf:"varint,2,opt,name=crc_encryptedticket" json:"crc_encryptedticket,omitempty"`
+	CbEncrypteduserdata           *uint32 `protobuf:"varint,3,opt,name=cb_encrypteduserdata" json:"cb_encrypteduserdata,omitempty"`
+	CbEncryptedAppownershipticket *uint32 `protobuf:"varint,4,opt,name=cb_encrypted_appownershipticket" json:"cb_encrypted_appownershipticket,omitempty"`
+	EncryptedTicket               []byte  `protobuf:"bytes,5,opt,name=encrypted_ticket" json:"encrypted_ticket,omitempty"`
+	XXX_unrecognized              []byte  `json:"-"`
+}
+
+func (m *EncryptedAppTicket) Reset()                    { *m = EncryptedAppTicket{} }
+func (m *EncryptedAppTicket) String() string            { return proto.CompactTextString(m) }
+func (*EncryptedAppTicket) ProtoMessage()               {}
+func (*EncryptedAppTicket) Descriptor() ([]byte, []int) { return app_ticket_fileDescriptor0, []int{0} }
+
+func (m *EncryptedAppTicket) GetTicketVersionNo() uint32 {
+	if m != nil && m.TicketVersionNo != nil {
+		return *m.TicketVersionNo
+	}
+	return 0
+}
+
+func (m *EncryptedAppTicket) GetCrcEncryptedticket() uint32 {
+	if m != nil && m.CrcEncryptedticket != nil {
+		return *m.CrcEncryptedticket
+	}
+	return 0
+}
+
+func (m *EncryptedAppTicket) GetCbEncrypteduserdata() uint32 {
+	if m != nil && m.CbEncrypteduserdata != nil {
+		return *m.CbEncrypteduserdata
+	}
+	return 0
+}
+
+func (m *EncryptedAppTicket) GetCbEncryptedAppownershipticket() uint32 {
+	if m != nil && m.CbEncryptedAppownershipticket != nil {
+		return *m.CbEncryptedAppownershipticket
+	}
+	return 0
+}
+
+func (m *EncryptedAppTicket) GetEncryptedTicket() []byte {
+	if m != nil {
+		return m.EncryptedTicket
+	}
+	return nil
+}
+
+func init() {
+	proto.RegisterType((*EncryptedAppTicket)(nil), "EncryptedAppTicket")
+}
+
+var app_ticket_fileDescriptor0 = []byte{
+	// 162 bytes of a gzipped FileDescriptorProto
+	0x1f, 0x8b, 0x08, 0x00, 0x00, 0x09, 0x6e, 0x88, 0x02, 0xff, 0xe2, 0x92, 0x4a, 0xcd, 0x4b, 0x2e,
+	0xaa, 0x2c, 0x28, 0x49, 0x4d, 0x89, 0x4f, 0x2c, 0x28, 0x88, 0x2f, 0xc9, 0x4c, 0xce, 0x4e, 0x2d,
+	0xd1, 0x2b, 0x28, 0xca, 0x2f, 0xc9, 0x57, 0x5a, 0xcb, 0xc8, 0x25, 0xe4, 0x0a, 0x93, 0x76, 0x2c,
+	0x28, 0x08, 0x01, 0x4b, 0x0a, 0x49, 0x72, 0x09, 0x42, 0x94, 0xc5, 0x97, 0xa5, 0x16, 0x15, 0x67,
+	0xe6, 0xe7, 0xc5, 0xe7, 0xe5, 0x4b, 0x30, 0x2a, 0x30, 0x6a, 0xf0, 0x0a, 0x49, 0x73, 0x09, 0x27,
+	0x17, 0x25, 0xc7, 0xc3, 0xcd, 0x84, 0xa8, 0x93, 0x60, 0x02, 0x4b, 0xca, 0x70, 0x89, 0x24, 0x27,
+	0x21, 0xe4, 0x4a, 0x8b, 0x53, 0x8b, 0x52, 0x12, 0x4b, 0x12, 0x25, 0x98, 0xc1, 0xb2, 0xea, 0x5c,
+	0xf2, 0xc8, 0xb2, 0x20, 0xd7, 0xe4, 0x97, 0xe7, 0x01, 0x2d, 0xc8, 0xc8, 0x2c, 0x80, 0x1a, 0xc3,
+	0x02, 0x56, 0x28, 0xc1, 0x25, 0x80, 0x50, 0x05, 0x95, 0x61, 0x05, 0xca, 0xf0, 0x38, 0xb1, 0x7a,
+	0x30, 0x36, 0x30, 0x32, 0x00, 0x02, 0x00, 0x00, 0xff, 0xff, 0x03, 0x8c, 0xdb, 0x92, 0xd3, 0x00,
+	0x00, 0x00,
+}

vendor/github.com/Philipp15b/go-steam/protocol/protobuf/base.pb.go

@@ -0,0 +1,613 @@
+// Code generated by protoc-gen-go.
+// source: steammessages_base.proto
+// DO NOT EDIT!
+
+package protobuf
+
+import proto "github.com/golang/protobuf/proto"
+import fmt "fmt"
+import math "math"
+import google_protobuf "github.com/golang/protobuf/protoc-gen-go/descriptor"
+
+// Reference imports to suppress errors if they are not otherwise used.
+var _ = proto.Marshal
+var _ = fmt.Errorf
+var _ = math.Inf
+
+type CMsgProtoBufHeader struct {
+	Steamid           *uint64 `protobuf:"fixed64,1,opt,name=steamid" json:"steamid,omitempty"`
+	ClientSessionid   *int32  `protobuf:"varint,2,opt,name=client_sessionid" json:"client_sessionid,omitempty"`
+	RoutingAppid      *uint32 `protobuf:"varint,3,opt,name=routing_appid" json:"routing_appid,omitempty"`
+	JobidSource       *uint64 `protobuf:"fixed64,10,opt,name=jobid_source,def=18446744073709551615" json:"jobid_source,omitempty"`
+	JobidTarget       *uint64 `protobuf:"fixed64,11,opt,name=jobid_target,def=18446744073709551615" json:"jobid_target,omitempty"`
+	TargetJobName     *string `protobuf:"bytes,12,opt,name=target_job_name" json:"target_job_name,omitempty"`
+	SeqNum            *int32  `protobuf:"varint,24,opt,name=seq_num" json:"seq_num,omitempty"`
+	Eresult           *int32  `protobuf:"varint,13,opt,name=eresult,def=2" json:"eresult,omitempty"`
+	ErrorMessage      *string `protobuf:"bytes,14,opt,name=error_message" json:"error_message,omitempty"`
+	Ip                *uint32 `protobuf:"varint,15,opt,name=ip" json:"ip,omitempty"`
+	AuthAccountFlags  *uint32 `protobuf:"varint,16,opt,name=auth_account_flags" json:"auth_account_flags,omitempty"`
+	TokenSource       *uint32 `protobuf:"varint,22,opt,name=token_source" json:"token_source,omitempty"`
+	AdminSpoofingUser *bool   `protobuf:"varint,23,opt,name=admin_spoofing_user" json:"admin_spoofing_user,omitempty"`
+	TransportError    *int32  `protobuf:"varint,17,opt,name=transport_error,def=1" json:"transport_error,omitempty"`
+	Messageid         *uint64 `protobuf:"varint,18,opt,name=messageid,def=18446744073709551615" json:"messageid,omitempty"`
+	PublisherGroupId  *uint32 `protobuf:"varint,19,opt,name=publisher_group_id" json:"publisher_group_id,omitempty"`
+	Sysid             *uint32 `protobuf:"varint,20,opt,name=sysid" json:"sysid,omitempty"`
+	TraceTag          *uint64 `protobuf:"varint,21,opt,name=trace_tag" json:"trace_tag,omitempty"`
+	WebapiKeyId       *uint32 `protobuf:"varint,25,opt,name=webapi_key_id" json:"webapi_key_id,omitempty"`
+	XXX_unrecognized  []byte  `json:"-"`
+}
+
+func (m *CMsgProtoBufHeader) Reset()                    { *m = CMsgProtoBufHeader{} }
+func (m *CMsgProtoBufHeader) String() string            { return proto.CompactTextString(m) }
+func (*CMsgProtoBufHeader) ProtoMessage()               {}
+func (*CMsgProtoBufHeader) Descriptor() ([]byte, []int) { return base_fileDescriptor0, []int{0} }
+
+const Default_CMsgProtoBufHeader_JobidSource uint64 = 18446744073709551615
+const Default_CMsgProtoBufHeader_JobidTarget uint64 = 18446744073709551615
+const Default_CMsgProtoBufHeader_Eresult int32 = 2
+const Default_CMsgProtoBufHeader_TransportError int32 = 1
+const Default_CMsgProtoBufHeader_Messageid uint64 = 18446744073709551615
+
+func (m *CMsgProtoBufHeader) GetSteamid() uint64 {
+	if m != nil && m.Steamid != nil {
+		return *m.Steamid
+	}
+	return 0
+}
+
+func (m *CMsgProtoBufHeader) GetClientSessionid() int32 {
+	if m != nil && m.ClientSessionid != nil {
+		return *m.ClientSessionid
+	}
+	return 0
+}
+
+func (m *CMsgProtoBufHeader) GetRoutingAppid() uint32 {
+	if m != nil && m.RoutingAppid != nil {
+		return *m.RoutingAppid
+	}
+	return 0
+}
+
+func (m *CMsgProtoBufHeader) GetJobidSource() uint64 {
+	if m != nil && m.JobidSource != nil {
+		return *m.JobidSource
+	}
+	return Default_CMsgProtoBufHeader_JobidSource
+}
+
+func (m *CMsgProtoBufHeader) GetJobidTarget() uint64 {
+	if m != nil && m.JobidTarget != nil {
+		return *m.JobidTarget
+	}
+	return Default_CMsgProtoBufHeader_JobidTarget
+}
+
+func (m *CMsgProtoBufHeader) GetTargetJobName() string {
+	if m != nil && m.TargetJobName != nil {
+		return *m.TargetJobName
+	}
+	return ""
+}
+
+func (m *CMsgProtoBufHeader) GetSeqNum() int32 {
+	if m != nil && m.SeqNum != nil {
+		return *m.SeqNum
+	}
+	return 0
+}
+
+func (m *CMsgProtoBufHeader) GetEresult() int32 {
+	if m != nil && m.Eresult != nil {
+		return *m.Eresult
+	}
+	return Default_CMsgProtoBufHeader_Eresult
+}
+
+func (m *CMsgProtoBufHeader) GetErrorMessage() string {
+	if m != nil && m.ErrorMessage != nil {
+		return *m.ErrorMessage
+	}
+	return ""
+}
+
+func (m *CMsgProtoBufHeader) GetIp() uint32 {
+	if m != nil && m.Ip != nil {
+		return *m.Ip
+	}
+	return 0
+}
+
+func (m *CMsgProtoBufHeader) GetAuthAccountFlags() uint32 {
+	if m != nil && m.AuthAccountFlags != nil {
+		return *m.AuthAccountFlags
+	}
+	return 0
+}
+
+func (m *CMsgProtoBufHeader) GetTokenSource() uint32 {
+	if m != nil && m.TokenSource != nil {
+		return *m.TokenSource
+	}
+	return 0
+}
+
+func (m *CMsgProtoBufHeader) GetAdminSpoofingUser() bool {
+	if m != nil && m.AdminSpoofingUser != nil {
+		return *m.AdminSpoofingUser
+	}
+	return false
+}
+
+func (m *CMsgProtoBufHeader) GetTransportError() int32 {
+	if m != nil && m.TransportError != nil {
+		return *m.TransportError
+	}
+	return Default_CMsgProtoBufHeader_TransportError
+}
+
+func (m *CMsgProtoBufHeader) GetMessageid() uint64 {
+	if m != nil && m.Messageid != nil {
+		return *m.Messageid
+	}
+	return Default_CMsgProtoBufHeader_Messageid
+}
+
+func (m *CMsgProtoBufHeader) GetPublisherGroupId() uint32 {
+	if m != nil && m.PublisherGroupId != nil {
+		return *m.PublisherGroupId
+	}
+	return 0
+}
+
+func (m *CMsgProtoBufHeader) GetSysid() uint32 {
+	if m != nil && m.Sysid != nil {
+		return *m.Sysid
+	}
+	return 0
+}
+
+func (m *CMsgProtoBufHeader) GetTraceTag() uint64 {
+	if m != nil && m.TraceTag != nil {
+		return *m.TraceTag
+	}
+	return 0
+}
+
+func (m *CMsgProtoBufHeader) GetWebapiKeyId() uint32 {
+	if m != nil && m.WebapiKeyId != nil {
+		return *m.WebapiKeyId
+	}
+	return 0
+}
+
+type CMsgMulti struct {
+	SizeUnzipped     *uint32 `protobuf:"varint,1,opt,name=size_unzipped" json:"size_unzipped,omitempty"`
+	MessageBody      []byte  `protobuf:"bytes,2,opt,name=message_body" json:"message_body,omitempty"`
+	XXX_unrecognized []byte  `json:"-"`
+}
+
+func (m *CMsgMulti) Reset()                    { *m = CMsgMulti{} }
+func (m *CMsgMulti) String() string            { return proto.CompactTextString(m) }
+func (*CMsgMulti) ProtoMessage()               {}
+func (*CMsgMulti) Descriptor() ([]byte, []int) { return base_fileDescriptor0, []int{1} }
+
+func (m *CMsgMulti) GetSizeUnzipped() uint32 {
+	if m != nil && m.SizeUnzipped != nil {
+		return *m.SizeUnzipped
+	}
+	return 0
+}
+
+func (m *CMsgMulti) GetMessageBody() []byte {
+	if m != nil {
+		return m.MessageBody
+	}
+	return nil
+}
+
+type CMsgProtobufWrapped struct {
+	MessageBody      []byte `protobuf:"bytes,1,opt,name=message_body" json:"message_body,omitempty"`
+	XXX_unrecognized []byte `json:"-"`
+}
+
+func (m *CMsgProtobufWrapped) Reset()                    { *m = CMsgProtobufWrapped{} }
+func (m *CMsgProtobufWrapped) String() string            { return proto.CompactTextString(m) }
+func (*CMsgProtobufWrapped) ProtoMessage()               {}
+func (*CMsgProtobufWrapped) Descriptor() ([]byte, []int) { return base_fileDescriptor0, []int{2} }
+
+func (m *CMsgProtobufWrapped) GetMessageBody() []byte {
+	if m != nil {
+		return m.MessageBody
+	}
+	return nil
+}
+
+type CMsgAuthTicket struct {
+	Estate           *uint32 `protobuf:"varint,1,opt,name=estate" json:"estate,omitempty"`
+	Eresult          *uint32 `protobuf:"varint,2,opt,name=eresult,def=2" json:"eresult,omitempty"`
+	Steamid          *uint64 `protobuf:"fixed64,3,opt,name=steamid" json:"steamid,omitempty"`
+	Gameid           *uint64 `protobuf:"fixed64,4,opt,name=gameid" json:"gameid,omitempty"`
+	HSteamPipe       *uint32 `protobuf:"varint,5,opt,name=h_steam_pipe" json:"h_steam_pipe,omitempty"`
+	TicketCrc        *uint32 `protobuf:"varint,6,opt,name=ticket_crc" json:"ticket_crc,omitempty"`
+	Ticket           []byte  `protobuf:"bytes,7,opt,name=ticket" json:"ticket,omitempty"`
+	XXX_unrecognized []byte  `json:"-"`
+}
+
+func (m *CMsgAuthTicket) Reset()                    { *m = CMsgAuthTicket{} }
+func (m *CMsgAuthTicket) String() string            { return proto.CompactTextString(m) }
+func (*CMsgAuthTicket) ProtoMessage()               {}
+func (*CMsgAuthTicket) Descriptor() ([]byte, []int) { return base_fileDescriptor0, []int{3} }
+
+const Default_CMsgAuthTicket_Eresult uint32 = 2
+
+func (m *CMsgAuthTicket) GetEstate() uint32 {
+	if m != nil && m.Estate != nil {
+		return *m.Estate
+	}
+	return 0
+}
+
+func (m *CMsgAuthTicket) GetEresult() uint32 {
+	if m != nil && m.Eresult != nil {
+		return *m.Eresult
+	}
+	return Default_CMsgAuthTicket_Eresult
+}
+
+func (m *CMsgAuthTicket) GetSteamid() uint64 {
+	if m != nil && m.Steamid != nil {
+		return *m.Steamid
+	}
+	return 0
+}
+
+func (m *CMsgAuthTicket) GetGameid() uint64 {
+	if m != nil && m.Gameid != nil {
+		return *m.Gameid
+	}
+	return 0
+}
+
+func (m *CMsgAuthTicket) GetHSteamPipe() uint32 {
+	if m != nil && m.HSteamPipe != nil {
+		return *m.HSteamPipe
+	}
+	return 0
+}
+
+func (m *CMsgAuthTicket) GetTicketCrc() uint32 {
+	if m != nil && m.TicketCrc != nil {
+		return *m.TicketCrc
+	}
+	return 0
+}
+
+func (m *CMsgAuthTicket) GetTicket() []byte {
+	if m != nil {
+		return m.Ticket
+	}
+	return nil
+}
+
+type CCDDBAppDetailCommon struct {
+	Appid                 *uint32 `protobuf:"varint,1,opt,name=appid" json:"appid,omitempty"`
+	Name                  *string `protobuf:"bytes,2,opt,name=name" json:"name,omitempty"`
+	Icon                  *string `protobuf:"bytes,3,opt,name=icon" json:"icon,omitempty"`
+	Logo                  *string `protobuf:"bytes,4,opt,name=logo" json:"logo,omitempty"`
+	LogoSmall             *string `protobuf:"bytes,5,opt,name=logo_small" json:"logo_small,omitempty"`
+	Tool                  *bool   `protobuf:"varint,6,opt,name=tool" json:"tool,omitempty"`
+	Demo                  *bool   `protobuf:"varint,7,opt,name=demo" json:"demo,omitempty"`
+	Media                 *bool   `protobuf:"varint,8,opt,name=media" json:"media,omitempty"`
+	CommunityVisibleStats *bool   `protobuf:"varint,9,opt,name=community_visible_stats" json:"community_visible_stats,omitempty"`
+	FriendlyName          *string `protobuf:"bytes,10,opt,name=friendly_name" json:"friendly_name,omitempty"`
+	Propagation           *string `protobuf:"bytes,11,opt,name=propagation" json:"propagation,omitempty"`
+	HasAdultContent       *bool   `protobuf:"varint,12,opt,name=has_adult_content" json:"has_adult_content,omitempty"`
+	XXX_unrecognized      []byte  `json:"-"`
+}
+
+func (m *CCDDBAppDetailCommon) Reset()                    { *m = CCDDBAppDetailCommon{} }
+func (m *CCDDBAppDetailCommon) String() string            { return proto.CompactTextString(m) }
+func (*CCDDBAppDetailCommon) ProtoMessage()               {}
+func (*CCDDBAppDetailCommon) Descriptor() ([]byte, []int) { return base_fileDescriptor0, []int{4} }
+
+func (m *CCDDBAppDetailCommon) GetAppid() uint32 {
+	if m != nil && m.Appid != nil {
+		return *m.Appid
+	}
+	return 0
+}
+
+func (m *CCDDBAppDetailCommon) GetName() string {
+	if m != nil && m.Name != nil {
+		return *m.Name
+	}
+	return ""
+}
+
+func (m *CCDDBAppDetailCommon) GetIcon() string {
+	if m != nil && m.Icon != nil {
+		return *m.Icon
+	}
+	return ""
+}
+
+func (m *CCDDBAppDetailCommon) GetLogo() string {
+	if m != nil && m.Logo != nil {
+		return *m.Logo
+	}
+	return ""
+}
+
+func (m *CCDDBAppDetailCommon) GetLogoSmall() string {
+	if m != nil && m.LogoSmall != nil {
+		return *m.LogoSmall
+	}
+	return ""
+}
+
+func (m *CCDDBAppDetailCommon) GetTool() bool {
+	if m != nil && m.Tool != nil {
+		return *m.Tool
+	}
+	return false
+}
+
+func (m *CCDDBAppDetailCommon) GetDemo() bool {
+	if m != nil && m.Demo != nil {
+		return *m.Demo
+	}
+	return false
+}
+
+func (m *CCDDBAppDetailCommon) GetMedia() bool {
+	if m != nil && m.Media != nil {
+		return *m.Media
+	}
+	return false
+}
+
+func (m *CCDDBAppDetailCommon) GetCommunityVisibleStats() bool {
+	if m != nil && m.CommunityVisibleStats != nil {
+		return *m.CommunityVisibleStats
+	}
+	return false
+}
+
+func (m *CCDDBAppDetailCommon) GetFriendlyName() string {
+	if m != nil && m.FriendlyName != nil {
+		return *m.FriendlyName
+	}
+	return ""
+}
+
+func (m *CCDDBAppDetailCommon) GetPropagation() string {
+	if m != nil && m.Propagation != nil {
+		return *m.Propagation
+	}
+	return ""
+}
+
+func (m *CCDDBAppDetailCommon) GetHasAdultContent() bool {
+	if m != nil && m.HasAdultContent != nil {
+		return *m.HasAdultContent
+	}
+	return false
+}
+
+type CMsgAppRights struct {
+	EditInfo                 *bool  `protobuf:"varint,1,opt,name=edit_info" json:"edit_info,omitempty"`
+	Publish                  *bool  `protobuf:"varint,2,opt,name=publish" json:"publish,omitempty"`
+	ViewErrorData            *bool  `protobuf:"varint,3,opt,name=view_error_data" json:"view_error_data,omitempty"`
+	Download                 *bool  `protobuf:"varint,4,opt,name=download" json:"download,omitempty"`
+	UploadCdkeys             *bool  `protobuf:"varint,5,opt,name=upload_cdkeys" json:"upload_cdkeys,omitempty"`
+	GenerateCdkeys           *bool  `protobuf:"varint,6,opt,name=generate_cdkeys" json:"generate_cdkeys,omitempty"`
+	ViewFinancials           *bool  `protobuf:"varint,7,opt,name=view_financials" json:"view_financials,omitempty"`
+	ManageCeg                *bool  `protobuf:"varint,8,opt,name=manage_ceg" json:"manage_ceg,omitempty"`
+	ManageSigning            *bool  `protobuf:"varint,9,opt,name=manage_signing" json:"manage_signing,omitempty"`
+	ManageCdkeys             *bool  `protobuf:"varint,10,opt,name=manage_cdkeys" json:"manage_cdkeys,omitempty"`
+	EditMarketing            *bool  `protobuf:"varint,11,opt,name=edit_marketing" json:"edit_marketing,omitempty"`
+	EconomySupport           *bool  `protobuf:"varint,12,opt,name=economy_support" json:"economy_support,omitempty"`
+	EconomySupportSupervisor *bool  `protobuf:"varint,13,opt,name=economy_support_supervisor" json:"economy_support_supervisor,omitempty"`
+	ManagePricing            *bool  `protobuf:"varint,14,opt,name=manage_pricing" json:"manage_pricing,omitempty"`
+	BroadcastLive            *bool  `protobuf:"varint,15,opt,name=broadcast_live" json:"broadcast_live,omitempty"`
+	XXX_unrecognized         []byte `json:"-"`
+}
+
+func (m *CMsgAppRights) Reset()                    { *m = CMsgAppRights{} }
+func (m *CMsgAppRights) String() string            { return proto.CompactTextString(m) }
+func (*CMsgAppRights) ProtoMessage()               {}
+func (*CMsgAppRights) Descriptor() ([]byte, []int) { return base_fileDescriptor0, []int{5} }
+
+func (m *CMsgAppRights) GetEditInfo() bool {
+	if m != nil && m.EditInfo != nil {
+		return *m.EditInfo
+	}
+	return false
+}
+
+func (m *CMsgAppRights) GetPublish() bool {
+	if m != nil && m.Publish != nil {
+		return *m.Publish
+	}
+	return false
+}
+
+func (m *CMsgAppRights) GetViewErrorData() bool {
+	if m != nil && m.ViewErrorData != nil {
+		return *m.ViewErrorData
+	}
+	return false
+}
+
+func (m *CMsgAppRights) GetDownload() bool {
+	if m != nil && m.Download != nil {
+		return *m.Download
+	}
+	return false
+}
+
+func (m *CMsgAppRights) GetUploadCdkeys() bool {
+	if m != nil && m.UploadCdkeys != nil {
+		return *m.UploadCdkeys
+	}
+	return false
+}
+
+func (m *CMsgAppRights) GetGenerateCdkeys() bool {
+	if m != nil && m.GenerateCdkeys != nil {
+		return *m.GenerateCdkeys
+	}
+	return false
+}
+
+func (m *CMsgAppRights) GetViewFinancials() bool {
+	if m != nil && m.ViewFinancials != nil {
+		return *m.ViewFinancials
+	}
+	return false
+}
+
+func (m *CMsgAppRights) GetManageCeg() bool {
+	if m != nil && m.ManageCeg != nil {
+		return *m.ManageCeg
+	}
+	return false
+}
+
+func (m *CMsgAppRights) GetManageSigning() bool {
+	if m != nil && m.ManageSigning != nil {
+		return *m.ManageSigning
+	}
+	return false
+}
+
+func (m *CMsgAppRights) GetManageCdkeys() bool {
+	if m != nil && m.ManageCdkeys != nil {
+		return *m.ManageCdkeys
+	}
+	return false
+}
+
+func (m *CMsgAppRights) GetEditMarketing() bool {
+	if m != nil && m.EditMarketing != nil {
+		return *m.EditMarketing
+	}
+	return false
+}
+
+func (m *CMsgAppRights) GetEconomySupport() bool {
+	if m != nil && m.EconomySupport != nil {
+		return *m.EconomySupport
+	}
+	return false
+}
+
+func (m *CMsgAppRights) GetEconomySupportSupervisor() bool {
+	if m != nil && m.EconomySupportSupervisor != nil {
+		return *m.EconomySupportSupervisor
+	}
+	return false
+}
+
+func (m *CMsgAppRights) GetManagePricing() bool {
+	if m != nil && m.ManagePricing != nil {
+		return *m.ManagePricing
+	}
+	return false
+}
+
+func (m *CMsgAppRights) GetBroadcastLive() bool {
+	if m != nil && m.BroadcastLive != nil {
+		return *m.BroadcastLive
+	}
+	return false
+}
+
+var E_MsgpoolSoftLimit = &proto.ExtensionDesc{
+	ExtendedType:  (*google_protobuf.MessageOptions)(nil),
+	ExtensionType: (*int32)(nil),
+	Field:         50000,
+	Name:          "msgpool_soft_limit",
+	Tag:           "varint,50000,opt,name=msgpool_soft_limit,def=32",
+}
+
+var E_MsgpoolHardLimit = &proto.ExtensionDesc{
+	ExtendedType:  (*google_protobuf.MessageOptions)(nil),
+	ExtensionType: (*int32)(nil),
+	Field:         50001,
+	Name:          "msgpool_hard_limit",
+	Tag:           "varint,50001,opt,name=msgpool_hard_limit,def=384",
+}
+
+func init() {
+	proto.RegisterType((*CMsgProtoBufHeader)(nil), "CMsgProtoBufHeader")
+	proto.RegisterType((*CMsgMulti)(nil), "CMsgMulti")
+	proto.RegisterType((*CMsgProtobufWrapped)(nil), "CMsgProtobufWrapped")
+	proto.RegisterType((*CMsgAuthTicket)(nil), "CMsgAuthTicket")
+	proto.RegisterType((*CCDDBAppDetailCommon)(nil), "CCDDBAppDetailCommon")
+	proto.RegisterType((*CMsgAppRights)(nil), "CMsgAppRights")
+	proto.RegisterExtension(E_MsgpoolSoftLimit)
+	proto.RegisterExtension(E_MsgpoolHardLimit)
+}
+
+var base_fileDescriptor0 = []byte{
+	// 906 bytes of a gzipped FileDescriptorProto
+	0x1f, 0x8b, 0x08, 0x00, 0x00, 0x09, 0x6e, 0x88, 0x02, 0xff, 0x8c, 0x54, 0x4d, 0x6f, 0x1c, 0x45,
+	0x10, 0x65, 0x77, 0xfd, 0x31, 0xdb, 0xde, 0x5d, 0xdb, 0x63, 0x27, 0xee, 0x98, 0x43, 0xa2, 0xbd,
+	0x80, 0x40, 0x72, 0xe2, 0x78, 0x1d, 0x1b, 0xdf, 0xfc, 0x71, 0xc8, 0xc5, 0x02, 0x21, 0x24, 0x8e,
+	0xad, 0x9e, 0x99, 0xda, 0xd9, 0xc6, 0x33, 0xdd, 0x4d, 0x77, 0x8f, 0xad, 0xcd, 0x89, 0x13, 0x57,
+	0xfe, 0x1a, 0xfc, 0x12, 0x6e, 0x88, 0x23, 0xd5, 0x35, 0xb3, 0x38, 0x04, 0x81, 0x72, 0x1a, 0x55,
+	0xd5, 0xeb, 0xaa, 0x57, 0xaf, 0xaa, 0x86, 0x71, 0x1f, 0x40, 0xd6, 0x35, 0x78, 0x2f, 0x4b, 0xf0,
+	0x22, 0x93, 0x1e, 0x8e, 0xac, 0x33, 0xc1, 0x1c, 0xbe, 0x28, 0x8d, 0x29, 0x2b, 0x78, 0x49, 0x56,
+	0xd6, 0xcc, 0x5f, 0x16, 0xe0, 0x73, 0xa7, 0x6c, 0x30, 0xae, 0x45, 0x4c, 0xff, 0x1c, 0xb0, 0xf4,
+	0xfa, 0xd6, 0x97, 0xdf, 0x44, 0xeb, 0xaa, 0x99, 0xbf, 0x05, 0x59, 0x80, 0x4b, 0xb7, 0xd9, 0x26,
+	0x25, 0x55, 0x05, 0xef, 0xbd, 0xe8, 0x7d, 0xbe, 0x91, 0x72, 0xb6, 0x93, 0x57, 0x0a, 0x74, 0x10,
+	0x1e, 0xeb, 0x28, 0xa3, 0x31, 0xd2, 0xc7, 0xc8, 0x7a, 0xfa, 0x84, 0x8d, 0x9d, 0x69, 0x82, 0xd2,
+	0xa5, 0x90, 0xd6, 0xa2, 0x7b, 0x80, 0xee, 0x71, 0xfa, 0x05, 0x1b, 0xfd, 0x60, 0x32, 0x55, 0x08,
+	0x6f, 0x1a, 0x97, 0x03, 0x67, 0x31, 0xcd, 0xc5, 0xfe, 0xf1, 0xf9, 0x6c, 0xf6, 0xe6, 0x6c, 0x36,
+	0x7b, 0x75, 0x76, 0x72, 0xf6, 0xea, 0xab, 0xd3, 0xd3, 0xe3, 0x37, 0xc7, 0xa7, 0x8f, 0xd8, 0x20,
+	0x5d, 0x09, 0x81, 0x6f, 0xfd, 0x0f, 0xf6, 0x80, 0x6d, 0xb7, 0x28, 0x81, 0x4f, 0x84, 0x96, 0x35,
+	0xf0, 0x11, 0xc2, 0x87, 0x44, 0x19, 0x7e, 0x14, 0xba, 0xa9, 0x39, 0x27, 0x62, 0x29, 0xdb, 0x04,
+	0x07, 0xbe, 0xa9, 0x02, 0x1f, 0x47, 0xc7, 0x45, 0xef, 0x75, 0x24, 0x0b, 0xce, 0x19, 0x27, 0x3a,
+	0xb5, 0xf8, 0x84, 0xde, 0x32, 0xd6, 0x57, 0x96, 0x6f, 0x13, 0xf1, 0x43, 0x96, 0xca, 0x26, 0x2c,
+	0x84, 0xcc, 0x73, 0xd3, 0x60, 0xbf, 0xf3, 0x4a, 0x96, 0x9e, 0xef, 0x50, 0x6c, 0x9f, 0x8d, 0x82,
+	0xb9, 0x03, 0xbd, 0x6a, 0xea, 0x29, 0x79, 0x3f, 0x65, 0x7b, 0xb2, 0xa8, 0x15, 0x7a, 0xad, 0x31,
+	0xf3, 0x28, 0x44, 0xe3, 0xc1, 0xf1, 0x03, 0x0c, 0x26, 0x98, 0x6e, 0x3b, 0x38, 0xa9, 0x31, 0xe4,
+	0x82, 0xa0, 0xda, 0x7c, 0xb7, 0x65, 0x73, 0x9c, 0x7e, 0xc6, 0x86, 0x1d, 0x0f, 0x94, 0x2d, 0x45,
+	0xef, 0xda, 0x7f, 0x34, 0x8d, 0x9c, 0x6c, 0x93, 0x55, 0xca, 0x2f, 0xc0, 0x89, 0x12, 0xe5, 0xb6,
+	0x02, 0x5f, 0xec, 0x51, 0xf5, 0x31, 0x5b, 0xf7, 0x4b, 0x8f, 0xe6, 0x3e, 0x99, 0xbb, 0x6c, 0x88,
+	0xf5, 0x72, 0x40, 0x2d, 0x4b, 0xfe, 0x24, 0xe6, 0x8c, 0x4d, 0x3f, 0x40, 0x26, 0xad, 0x12, 0x77,
+	0xb0, 0x8c, 0x0f, 0x9f, 0x45, 0xe4, 0xf4, 0x9c, 0x0d, 0xe3, 0xe4, 0x6f, 0x51, 0x20, 0x15, 0x31,
+	0x5e, 0xbd, 0x03, 0xd1, 0xe8, 0x77, 0xca, 0x5a, 0x68, 0xc7, 0x4e, 0x0d, 0x77, 0x0c, 0x45, 0x66,
+	0x8a, 0x25, 0x8d, 0x7c, 0x34, 0xfd, 0x92, 0xed, 0xfd, 0xbd, 0x33, 0xb8, 0x55, 0xdf, 0x3b, 0x19,
+	0x9f, 0xfc, 0x0b, 0xdc, 0x23, 0xf0, 0x2f, 0x3d, 0x36, 0x89, 0xe8, 0x4b, 0x14, 0xf5, 0x3b, 0x95,
+	0xdf, 0x41, 0x48, 0x27, 0x6c, 0x03, 0x7c, 0x90, 0x01, 0xba, 0x2a, 0xef, 0x4d, 0x2a, 0x16, 0x18,
+	0xc7, 0x49, 0xbd, 0xb7, 0x81, 0x03, 0xda, 0x40, 0x7c, 0x54, 0xe2, 0xb4, 0xd1, 0x5e, 0x23, 0x1b,
+	0xab, 0x2d, 0x04, 0x41, 0x84, 0x55, 0x16, 0xf8, 0x7a, 0x97, 0x8a, 0x05, 0x2a, 0x22, 0x72, 0x97,
+	0xf3, 0x0d, 0xf2, 0xe1, 0xcb, 0xd6, 0xc7, 0x37, 0x89, 0xd1, 0x1f, 0x3d, 0xb6, 0x7f, 0x7d, 0x7d,
+	0x73, 0x73, 0x75, 0x69, 0xed, 0x0d, 0x04, 0xa9, 0xaa, 0x6b, 0x53, 0xd7, 0x46, 0x47, 0x29, 0xdb,
+	0x15, 0x6e, 0x69, 0x8d, 0xd8, 0x1a, 0xed, 0x57, 0x9f, 0x76, 0x04, 0x2d, 0x95, 0x1b, 0x4d, 0x6c,
+	0xc8, 0xaa, 0x4c, 0x69, 0x88, 0xcb, 0x30, 0x56, 0x8d, 0x96, 0xf0, 0xb5, 0xac, 0x2a, 0x62, 0x42,
+	0x88, 0x60, 0x4c, 0x45, 0x1c, 0x92, 0x68, 0x15, 0x50, 0x1b, 0x62, 0x90, 0xc4, 0x42, 0x35, 0x14,
+	0x4a, 0xf2, 0x84, 0xcc, 0xe7, 0xec, 0x20, 0x47, 0x06, 0x8d, 0x56, 0x61, 0x29, 0xee, 0x95, 0x57,
+	0x59, 0x05, 0x22, 0x0a, 0xe4, 0xf9, 0x90, 0x00, 0x38, 0x9d, 0xb9, 0xc3, 0xeb, 0x2b, 0xaa, 0x65,
+	0xbb, 0xf2, 0x8c, 0x4a, 0xec, 0xb1, 0x2d, 0xbc, 0x62, 0x2b, 0x4b, 0x19, 0xf0, 0x22, 0xe9, 0x6c,
+	0x86, 0xe9, 0x33, 0xb6, 0xbb, 0x90, 0x5e, 0xc8, 0x02, 0xe5, 0x14, 0x48, 0x38, 0xe0, 0xd1, 0xd2,
+	0x89, 0x24, 0xd3, 0xdf, 0xfb, 0x6c, 0x4c, 0xa3, 0xb0, 0xf6, 0x5b, 0x55, 0x2e, 0x82, 0x8f, 0xdb,
+	0x82, 0x3c, 0x82, 0x50, 0x7a, 0x6e, 0xa8, 0xeb, 0x24, 0x0a, 0xdf, 0xed, 0x1a, 0x35, 0x9e, 0xc4,
+	0x8b, 0xbb, 0x57, 0xf0, 0xd0, 0x2e, 0xaf, 0x28, 0x64, 0x90, 0xa4, 0x41, 0x92, 0xee, 0xb0, 0xa4,
+	0x30, 0x0f, 0xba, 0x32, 0xb2, 0x9d, 0x09, 0xf1, 0x6c, 0x6c, 0xb4, 0x45, 0x5e, 0xe0, 0xae, 0x79,
+	0x92, 0x82, 0x32, 0x94, 0xa0, 0xc1, 0xe1, 0xc4, 0x57, 0x81, 0x8d, 0x7f, 0xa4, 0xc6, 0xa3, 0x91,
+	0x3a, 0x57, 0xb2, 0xf2, 0x9d, 0x40, 0x28, 0x68, 0x2d, 0x75, 0xdc, 0xa4, 0x1c, 0xca, 0x4e, 0xa5,
+	0xa7, 0x6c, 0xd2, 0xf9, 0xbc, 0x2a, 0x35, 0x9e, 0xd9, 0xa3, 0x38, 0x2b, 0x6c, 0x9b, 0x9b, 0xad,
+	0xe0, 0xd4, 0x5a, 0x2d, 0x1d, 0x8e, 0x3e, 0xc2, 0xb7, 0x56, 0x35, 0x01, 0x65, 0x31, 0xf5, 0x52,
+	0xf8, 0xc6, 0xc6, 0xb3, 0x6c, 0xd5, 0x49, 0xa7, 0xec, 0xf0, 0x83, 0x40, 0xfc, 0x82, 0xc3, 0x81,
+	0xe0, 0xd1, 0x8e, 0x3f, 0xe0, 0x60, 0x9d, 0xca, 0x63, 0xd2, 0xc9, 0xca, 0x9f, 0x39, 0xec, 0x3b,
+	0x97, 0x3e, 0x88, 0x4a, 0xdd, 0x03, 0xfd, 0x4c, 0x92, 0x8b, 0x4b, 0x96, 0xd6, 0xbe, 0xc4, 0xdf,
+	0x42, 0x85, 0xbf, 0x8c, 0x79, 0x0c, 0xd5, 0x2a, 0xa4, 0xcf, 0x8f, 0xda, 0xff, 0xf2, 0xd1, 0xea,
+	0xbf, 0x7c, 0x74, 0xdb, 0xde, 0xcd, 0xd7, 0x36, 0x0e, 0xd2, 0xf3, 0x5f, 0x7f, 0x1e, 0xd0, 0x3f,
+	0xa2, 0x7f, 0xf2, 0xfa, 0xe2, 0xea, 0x31, 0xc5, 0x42, 0xba, 0xe2, 0x63, 0x53, 0xfc, 0xd6, 0xa5,
+	0x18, 0x9c, 0x9c, 0xcf, 0xae, 0xd6, 0xdf, 0xf6, 0x7e, 0xea, 0x7d, 0xf2, 0x57, 0x00, 0x00, 0x00,
+	0xff, 0xff, 0x66, 0x1a, 0xa6, 0xfc, 0x29, 0x06, 0x00, 0x00,
+}