Release v1.3.1

Refactor message.Extra to a map[string][]interface{} to have a bit more flexibility
for stuffing extra stuff.

For attached files from slack, files < 1MB size get downloaded (in memory), and get
put into Extra["file"][]config.FileInfo (containing a pointer to the buffer and
the filename). This is not async so slack channels with lots of attached files
may suffer a slowdown. (the download timeout is set at 5 seconds).

.github/ISSUE_TEMPLATE.md

@@ -1,3 +1,5 @@
+If you have a configuration problem, please first try to create a basic configuration following the instructions on [the wiki](https://github.com/42wim/matterbridge/wiki/How-to-create-your-config) before filing an issue.
+
 Please answer the following questions. 
 
 ### Which version of matterbridge are you using?

.travis.yml

@@ -0,0 +1,49 @@
+language: go
+go:
+    #- 1.7.x
+    - 1.9.x
+      # - tip
+
+# we have everything vendored
+install: true
+
+env:
+    - GOOS=linux GOARCH=amd64
+      #    - GOOS=windows GOARCH=amd64
+      #- GOOS=linux GOARCH=arm
+
+matrix:
+  # It's ok if our code fails on unstable development versions of Go.
+  allow_failures:
+    - go: tip
+  # Don't wait for tip tests to finish. Mark the test run green if the
+  # tests pass on the stable versions of Go.
+  fast_finish: true
+
+notifications:
+      email: false
+
+before_script:
+  - MY_VERSION=$(git describe --tags)
+  - GO_FILES=$(find . -iname '*.go' | grep -v /vendor/)  # All the .go files, excluding vendor/
+  - PKGS=$(go list ./... | grep -v /vendor/)             # All the import paths, excluding vendor/
+#  - go get github.com/golang/lint/golint                 # Linter
+  - go get honnef.co/go/tools/cmd/megacheck              # Badass static analyzer/linter
+
+# Anything in before_script: that returns a nonzero exit code will
+# flunk the build and immediately stop. It's sorta like having
+# set -e enabled in bash. 
+script:
+  - test -z $(gofmt -s -l $GO_FILES)  # Fail if a .go file hasn't been formatted with gofmt
+  - go test -v -race $PKGS            # Run all the tests with the race detector enabled
+  - go vet $PKGS                      # go vet is the official Go static analyzer
+  - megacheck $PKGS                   # "go vet on steroids" + linter
+  - /bin/bash ci/bintray.sh
+  #- golint -set_exit_status $PKGS     # one last linter
+
+deploy:
+  provider: bintray
+  file: ci/deploy.json
+  user: 42wim
+  key:
+     secure: "CeXXe6JOmt7HYR81MdWLua0ltQHhDdkIeRGBFbgd7hkb1wi8eF9DgpAcQrTso8NIlHNZmSAP46uhFgsRvkuezzX0ygalZ7DCJyAyn3sAMEh+UQSHV1WGThRehTtidqRGjetzsIGSwdrJOWil+XTfbO1Z8DGzfakhSuAZka8CM4BAoe3YeP9rYK8h+84x0GHfczvsLtXZ3mWLvQuwe4pK6+ItBCUg0ae7O7ZUpWHy0xQQkkWztY/6RAzXfaG7DuGjIw+20fhx3WOXRNpHCtZ6Bc3qERCpk0s1HhlQWlrN9wDaFTBWYwlvSnNgvxxMbNXJ6RrRJ0l0bA7FUswYwyroxhzrGLdzWDg8dHaQkypocngdalfhpsnoO9j3ApJhomUFJ3UoEq5nOGRUrKn8MPi+dP0zE4kNQ3e4VNa1ufNrvfpWolMg3xh8OXuhQdD5wIM5zFAbRJLqWSCVAjPq4DDPecmvXBOlIial7oa312lN5qnBnUjvAcxszZ+FUyDHT1Grxzna4tMwxY9obPzZUzm7359AOCCwIQFVB8GLqD2nwIstcXS0zGRz+fhviPipHuBa02q5bGUZwmkvrSNab0s8Jo7pCrel2Rz3nWPKaiCfq2WjbW1CLheSMkOQrjsdUd1hhbqNWFPUjJPInTc77NAKCfm5runv5uyowRLh4NNd0sI="

README-0.6.md

@@ -1,115 +0,0 @@
-# matterbridge
-
-Simple bridge between mattermost, IRC, XMPP, Gitter and Slack
-
-* Relays public channel messages between mattermost, IRC, XMPP, Gitter and Slack. Pick and mix.
-* Supports multiple channels.
-* Matterbridge can also work with private groups on your mattermost.
-
-Look at [matterbridge.conf.sample] (https://github.com/42wim/matterbridge/blob/master/matterbridge.conf.sample) for documentation and an example.
-
-## Changelog
-Since v0.6.1 support for XMPP, Gitter and Slack is added. More details in [changelog.md] (https://github.com/42wim/matterbridge/blob/master/changelog.md)
-
-## Requirements:
-Accounts to one of the supported bridges
-* [Mattermost] (https://github.com/mattermost/platform/)
-* [IRC] (http://www.mirc.com/servers.html)
-* [XMPP] (https://jabber.org)
-* [Gitter] (https://gitter.im)
-* [Slack] (https://www.slack.com)
-
-## binaries
-Binaries can be found [here] (https://github.com/42wim/matterbridge/releases/)
-* For use with mattermost 3.3.0+ [v0.6.1](https://github.com/42wim/matterircd/releases/tag/v0.6.1)
-* For use with mattermost 3.0.0-3.2.0 [v0.5.0](https://github.com/42wim/matterircd/releases/tag/v0.5.0)
-
-
-## Docker
-Create your matterbridge.conf file locally eg in ```/tmp/matterbridge.conf```
-
-```
-docker run -ti -v /tmp/matterbridge.conf:/matterbridge.conf 42wim/matterbridge:0.6.1
-```
-
-## Compatibility
-### Mattermost 
-* Matterbridge v0.6.1 works with mattermost 3.3.0 and higher [3.3.0 release](https://github.com/mattermost/platform/releases/tag/v3.3.0)
-* Matterbridge v0.5.0 works with mattermost 3.0.0 - 3.2.0 [3.2.0 release](https://github.com/mattermost/platform/releases/tag/v3.2.0)
-
-
-#### Webhooks version
-* Configured incoming/outgoing [webhooks](https://www.mattermost.org/webhooks/) on your mattermost instance.
-
-#### Plus (API) version
-* A dedicated user(bot) on your mattermost instance.
-
-
-## building
-Go 1.6+ is required. Make sure you have [Go](https://golang.org/doc/install) properly installed, including setting up your [GOPATH] (https://golang.org/doc/code.html#GOPATH)
-
-```
-cd $GOPATH
-go get github.com/42wim/matterbridge
-```
-
-You should now have matterbridge binary in the bin directory:
-
-```
-$ ls bin/
-matterbridge
-```
-
-## running
-1) Copy the matterbridge.conf.sample to matterbridge.conf in the same directory as the matterbridge binary.  
-2) Edit matterbridge.conf with the settings for your environment. See below for more config information.  
-3) Now you can run matterbridge. 
-
-```
-Usage of ./matterbridge:
-  -conf string
-        config file (default "matterbridge.conf")
-  -debug
-        enable debug
-  -plus
-        running using API instead of webhooks (deprecated, set Plus flag in [general] config)
-  -version
-        show version
-```
-
-## config
-### matterbridge
-matterbridge looks for matterbridge.conf in current directory. (use -conf to specify another file)
-
-Look at [matterbridge.conf.sample] (https://github.com/42wim/matterbridge/blob/master/matterbridge.conf.sample) for an example.
-
-### mattermost
-#### webhooks version
-You'll have to configure the incoming and outgoing webhooks. 
-
-* incoming webhooks
-Go to "account settings" - integrations - "incoming webhooks".  
-Choose a channel at "Add a new incoming webhook", this will create a webhook URL right below.  
-This URL should be set in the matterbridge.conf in the [mattermost] section (see above)  
-
-* outgoing webhooks
-Go to "account settings" - integrations - "outgoing webhooks".  
-Choose a channel (the same as the one from incoming webhooks) and fill in the address and port of the server matterbridge will run on.  
-
-e.g. http://192.168.1.1:9999 (192.168.1.1:9999 is the BindAddress specified in [mattermost] section of matterbridge.conf)
-
-#### plus version
-You'll have to create a new dedicated user on your mattermost instance.
-Specify the login and password in [mattermost] section of matterbridge.conf
-
-## FAQ
-Please look at [matterbridge.conf.sample] (https://github.com/42wim/matterbridge/blob/master/matterbridge.conf.sample) for more information first. 
-### Mattermost doesn't show the IRC nicks
-If you're running the webhooks version, this can be fixed by either:
-* enabling "override usernames". See [mattermost documentation](http://docs.mattermost.com/developer/webhooks-incoming.html#enabling-incoming-webhooks)
-* setting ```PrefixMessagesWithNick``` to ```true``` in ```mattermost``` section of your matterbridge.conf.
-
-If you're running the plus version you'll need to:
-* setting ```PrefixMessagesWithNick``` to ```true``` in ```mattermost``` section of your matterbridge.conf.
-
-Also look at the ```RemoteNickFormat``` setting.

README.md

@@ -1,17 +1,24 @@
 # matterbridge
-[![Gitter](https://img.shields.io/gitter/room/nwjs/nw.js.svg)](https://gitter.im/42wim/matterbridge) [![Join the IRC chat at https://webchat.freenode.net/?channels=matterbridgechat](https://img.shields.io/badge/IRC-matterbridgechat-green.svg)](https://webchat.freenode.net/?channels=matterbridgechat) [![Discord](https://img.shields.io/badge/discord-matterbridge-green.svg)](https://discord.gg/AkKPtrQ) [![Matrix](https://img.shields.io/badge/matrix-matterbridge-green.svg)](https://riot.im/app/#/room/#matterbridge:matrix.org)
+Click on one of the badges below to join the chat   
+
+[![Gitter](https://img.shields.io/gitter/room/nwjs/nw.js.svg?colorB=42f4242)](https://gitter.im/42wim/matterbridge) [![Join the IRC chat at https://webchat.freenode.net/?channels=matterbridgechat](https://img.shields.io/badge/IRC-matterbridgechat-green.svg?colorB=42f4242)](https://webchat.freenode.net/?channels=matterbridgechat) [![Discord](https://img.shields.io/badge/discord-matterbridge-green.svg?colorB=42f4242)](https://discord.gg/AkKPtrQ) [![Matrix](https://img.shields.io/badge/matrix-matterbridge-green.svg?colorB=42f4242)](https://riot.im/app/#/room/#matterbridge:matrix.org) [![Slack](https://img.shields.io/badge/slack-matterbridgechat-green.svg?colorB=42f4242)](https://join.slack.com/matterbridgechat/shared_invite/MjEwODMxNjU1NDMwLTE0OTk2MTU3NTMtMzZkZmRiNDZhOA) [![Mattermost](https://img.shields.io/badge/mattermost-matterbridge-green.svg?colorB=42f4242)](https://framateam.org/signup_user_complete/?id=tfqm33ggop8x3qgu4boeieta6e) ![Xmpp](https://img.shields.io/badge/xmpp-matterbridge@muc.im.koderoot.net-green.svg?colorB=42f4242)
+
+[![Download stable](https://img.shields.io/github/release/42wim/matterbridge.svg?label=download%20stable)](https://github.com/42wim/matterbridge/releases/latest) [![Download dev](https://img.shields.io/bintray/v/42wim/nightly/Matterbridge.svg?label=download%20dev&colorB=007ec6)](https://bintray.com/42wim/nightly/Matterbridge/_latestVersion)
 
 ![matterbridge.gif](https://s15.postimg.org/qpjhp6y3f/matterbridge.gif)
 
-Simple bridge between Mattermost, IRC, XMPP, Gitter, Slack, Discord, Telegram, Rocket.Chat, Hipchat(via xmpp) and Matrix with REST API.
+Simple bridge between Mattermost, IRC, XMPP, Gitter, Slack, Discord, Telegram, Rocket.Chat, Hipchat(via xmpp), Matrix and Steam.
+Has a REST API.
 
 # Table of Contents
  * [Features](#features)
  * [Requirements](#requirements)
+ * [Screenshots](https://github.com/42wim/matterbridge/wiki/)
  * [Installing](#installing)
    * [Binaries](#binaries)
    * [Building](#building)
  * [Configuration](#configuration)
+   * [Howto](https://github.com/42wim/matterbridge/wiki/How-to-create-your-config)
    * [Examples](#examples) 
  * [Running](#running)
    * [Docker](#docker)
@@ -20,15 +27,17 @@ Simple bridge between Mattermost, IRC, XMPP, Gitter, Slack, Discord, Telegram, R
  * [Thanks](#thanks)
 
 # Features
-* Relays public channel messages between multiple mattermost, IRC, XMPP, Gitter, Slack, Discord, Telegram, Rocket.Chat, Hipchat (via xmpp) and Matrix. Pick and mix.
-* Matterbridge can also work with private groups on your mattermost/slack.
+* Relays public channel messages between multiple mattermost, IRC, XMPP, Gitter, Slack, Discord, Telegram, Rocket.Chat, Hipchat (via xmpp), Matrix and Steam. 
+  Pick and mix.
+* Support private groups on your mattermost/slack.
 * Allow for bridging the same bridges, which means you can eg bridge between multiple mattermosts.
 * The bridge is now a gateway which has support multiple in and out bridges. (and supports multiple gateways).
+* Edits and delete messages across bridges that support it (mattermost,slack,discord,gitter,telegram)
 * REST API to read/post messages to bridges (WIP).
 
 # Requirements
 Accounts to one of the supported bridges
-* [Mattermost](https://github.com/mattermost/platform/) 3.5.x - 3.10.x
+* [Mattermost](https://github.com/mattermost/platform/) 3.8.x - 3.10.x, 4.x
 * [IRC](http://www.mirc.com/servers.html)
 * [XMPP](https://jabber.org)
 * [Gitter](https://gitter.im)
@@ -38,14 +47,18 @@ Accounts to one of the supported bridges
 * [Hipchat](https://www.hipchat.com)
 * [Rocket.chat](https://rocket.chat)
 * [Matrix](https://matrix.org)
+* [Steam](https://store.steampowered.com/)
+
+# Screenshots
+See https://github.com/42wim/matterbridge/wiki
 
 # Installing
 ## Binaries
-Binaries can be found [here] (https://github.com/42wim/matterbridge/releases/)
-* Latest stable release [v0.15.0](https://github.com/42wim/matterbridge/releases/latest)
+* Latest stable release [v1.3.1](https://github.com/42wim/matterbridge/releases/latest)
+* Development releases (follows master) can be downloaded [here](https://dl.bintray.com/42wim/nightly/)  
 
 ## Building
-Go 1.6+ is required. Make sure you have [Go](https://golang.org/doc/install) properly installed, including setting up your [GOPATH] (https://golang.org/doc/code.html#GOPATH)
+Go 1.7+ is required. Make sure you have [Go](https://golang.org/doc/install) properly installed, including setting up your [GOPATH] (https://golang.org/doc/code.html#GOPATH)
 
 ```
 cd $GOPATH
@@ -60,10 +73,13 @@ matterbridge
 ```
 
 # Configuration
-* [matterbridge.toml.sample](https://github.com/42wim/matterbridge/blob/master/matterbridge.toml.sample) for documentation and an example.
-* [matterbridge.toml.simple](https://github.com/42wim/matterbridge/blob/master/matterbridge.toml.simple) for a simple example.
+## Basic configuration
+See [howto](https://github.com/42wim/matterbridge/wiki/How-to-create-your-config) for a step by step walkthrough for creating your configuration.
 
-## Examples
+## Advanced configuration
+* [matterbridge.toml.sample](https://github.com/42wim/matterbridge/blob/master/matterbridge.toml.sample) for documentation and an example.
+
+## Examples 
 ### Bridge mattermost (off-topic) - irc (#testing)
 ```
 [irc]
@@ -73,12 +89,12 @@ matterbridge
 
 [mattermost]
     [mattermost.work]
-    useAPI=true
     Server="yourmattermostserver.tld"
     Team="yourteam"
     Login="yourlogin"
     Password="yourpass"
     PrefixMessagesWithNick=true
+    RemoteNickFormat="[{PROTOCOL}] <{NICK}> "
 
 [[gateway]]
 name="mygateway"
@@ -96,7 +112,6 @@ enable=true
 ```
 [slack]
 [slack.test]
-useAPI=true
 Token="yourslacktoken"
 PrefixMessagesWithNick=true
 
@@ -122,11 +137,8 @@ RemoteNickFormat="[{PROTOCOL}/{BRIDGE}] <{NICK}> "
 ```
 
 # Running
-1) Copy the matterbridge.toml.sample to matterbridge.toml 
-2) Edit matterbridge.toml with the settings for your environment. 
-3) Now you can run matterbridge.  (```./matterbridge```)   
 
-(Matterbridge will only look for the config file in your current directory, if it isn't there specify -conf "/path/toyour/matterbridge.toml")
+See [howto](https://github.com/42wim/matterbridge/wiki/How-to-create-your-config) for a step by step walkthrough for creating your configuration.
 
 ```
 Usage of ./matterbridge:
@@ -151,18 +163,11 @@ See [changelog.md](https://github.com/42wim/matterbridge/blob/master/changelog.m
 
 # FAQ
 
-Please look at [matterbridge.toml.sample](https://github.com/42wim/matterbridge/blob/master/matterbridge.toml.sample) for more information first.
-
-## Mattermost doesn't show the IRC nicks
-If you're running the webhooks version, this can be fixed by either:
-* enabling "override usernames". See [mattermost documentation](http://docs.mattermost.com/developer/webhooks-incoming.html#enabling-incoming-webhooks)
-* setting ```PrefixMessagesWithNick``` to ```true``` in ```mattermost``` section of your matterbridge.toml.
-
-If you're running the API version you'll need to:
-* setting ```PrefixMessagesWithNick``` to ```true``` in ```mattermost``` section of your matterbridge.toml.
-
-Also look at the ```RemoteNickFormat``` setting.
+See [FAQ](https://github.com/42wim/matterbridge/wiki/FAQ)
 
+Want to tip ? 
+* eth: 0xb3f9b5387c66ad6be892bcb7bbc67862f3abc16f
+* btc: 1N7cKHj5SfqBHBzDJ6kad4BzeqUBBS2zhs
 
 # Thanks
 Matterbridge wouldn't exist without these libraries:
@@ -174,6 +179,6 @@ Matterbridge wouldn't exist without these libraries:
 * mattermost - https://github.com/mattermost/platform
 * matrix - https://github.com/matrix-org/gomatrix
 * slack - https://github.com/nlopes/slack
+* steam - https://github.com/Philipp15b/go-steam
 * telegram - https://github.com/go-telegram-bot-api/telegram-bot-api
 * xmpp - https://github.com/mattn/go-xmpp
-

bridge/api/api.go

@@ -61,16 +61,20 @@ func (b *Api) Disconnect() error {
 	return nil
 
 }
-func (b *Api) JoinChannel(channel string) error {
+func (b *Api) JoinChannel(channel config.ChannelInfo) error {
 	return nil
 
 }
 
-func (b *Api) Send(msg config.Message) error {
+func (b *Api) Send(msg config.Message) (string, error) {
 	b.Lock()
 	defer b.Unlock()
+	// ignore delete messages
+	if msg.Event == config.EVENT_MSG_DELETE {
+		return "", nil
+	}
 	b.Messages.Enqueue(&msg)
-	return nil
+	return "", nil
 }
 
 func (b *Api) handlePostMessage(c echo.Context) error {

bridge/bridge.go

@@ -19,9 +19,9 @@ import (
 )
 
 type Bridger interface {
-	Send(msg config.Message) error
+	Send(msg config.Message) (string, error)
 	Connect() error
-	JoinChannel(channel string) error
+	JoinChannel(channel config.ChannelInfo) error
 	Disconnect() error
 }
 
@@ -88,23 +88,14 @@ func New(cfg *config.Config, bridge *config.Bridge, c chan config.Message) *Brid
 
 func (b *Bridge) JoinChannels() error {
 	err := b.joinChannels(b.Channels, b.Joined)
-	if err != nil {
-		return err
-	}
-	return nil
+	return err
 }
 
 func (b *Bridge) joinChannels(channels map[string]config.ChannelInfo, exists map[string]bool) error {
-	mychannel := ""
 	for ID, channel := range channels {
 		if !exists[ID] {
-			mychannel = channel.Name
 			log.Infof("%s: joining %s (%s)", b.Account, channel.Name, ID)
-			if b.Protocol == "irc" && channel.Options.Key != "" {
-				log.Debugf("using key %s for channel %s", channel.Options.Key, channel.Name)
-				mychannel = mychannel + " " + channel.Options.Key
-			}
-			err := b.JoinChannel(mychannel)
+			err := b.JoinChannel(channel)
 			if err != nil {
 				return err
 			}

bridge/config/config.go

@@ -13,6 +13,8 @@ const (
 	EVENT_JOIN_LEAVE      = "join_leave"
 	EVENT_FAILURE         = "failure"
 	EVENT_REJOIN_CHANNELS = "rejoin_channels"
+	EVENT_USER_ACTION     = "user_action"
+	EVENT_MSG_DELETE      = "msg_delete"
 )
 
 type Message struct {
@@ -26,6 +28,13 @@ type Message struct {
 	Protocol  string    `json:"protocol"`
 	Gateway   string    `json:"gateway"`
 	Timestamp time.Time `json:"timestamp"`
+	ID        string    `json:"id"`
+	Extra     map[string][]interface{}
+}
+
+type FileInfo struct {
+	Name string
+	Data *[]byte
 }
 
 type ChannelInfo struct {
@@ -33,7 +42,6 @@ type ChannelInfo struct {
 	Account     string
 	Direction   string
 	ID          string
-	GID         map[string]bool
 	SameChannel map[string]bool
 	Options     ChannelOptions
 }
@@ -42,6 +50,7 @@ type Protocol struct {
 	AuthCode               string // steam
 	BindAddress            string // mattermost, slack // DEPRECATED
 	Buffer                 int    // api
+	Charset                string // irc
 	EditSuffix             string // mattermost, slack, discord, telegram, gitter
 	EditDisable            bool   // mattermost, slack, discord, telegram, gitter
 	IconURL                string // mattermost, slack
@@ -54,6 +63,7 @@ type Protocol struct {
 	Nick                   string // all protocols
 	NickFormatter          string // mattermost, slack
 	NickServNick           string // IRC
+	NickServUsername       string // IRC
 	NickServPassword       string // IRC
 	NicksPerRow            int    // mattermost, slack
 	NoHomeServerSuffix     bool   // matrix
@@ -77,13 +87,16 @@ type Protocol struct {
 	UseSASL                bool   // IRC
 	UseTLS                 bool   // IRC
 	UseFirstName           bool   // telegram
+	UseUserName            bool   // discord
+	UseInsecureURL         bool   // telegram
 	WebhookBindAddress     string // mattermost, slack
 	WebhookURL             string // mattermost, slack
 	WebhookUse             string // mattermost, slack, discord
 }
 
 type ChannelOptions struct {
-	Key string // irc
+	Key        string // irc
+	WebhookURL string // discord
 }
 
 type Bridge struct {
@@ -208,7 +221,7 @@ func Deprecated(cfg Protocol, account string) bool {
 		log.Printf("ERROR: %s BindAddress is deprecated, you need to change it to WebhookBindAddress.", account)
 	} else if cfg.URL != "" {
 		log.Printf("ERROR: %s URL is deprecated, you need to change it to WebhookURL.", account)
-	} else if cfg.UseAPI == true {
+	} else if cfg.UseAPI {
 		log.Printf("ERROR: %s UseAPI is deprecated, it's enabled by default, please remove it from your config file.", account)
 	} else {
 		return false

bridge/discord/discord.go

@@ -10,17 +10,18 @@ import (
 )
 
 type bdiscord struct {
-	c             *discordgo.Session
-	Config        *config.Protocol
-	Remote        chan config.Message
-	Account       string
-	Channels      []*discordgo.Channel
-	Nick          string
-	UseChannelID  bool
-	userMemberMap map[string]*discordgo.Member
-	guildID       string
-	webhookID     string
-	webhookToken  string
+	c              *discordgo.Session
+	Config         *config.Protocol
+	Remote         chan config.Message
+	Account        string
+	Channels       []*discordgo.Channel
+	Nick           string
+	UseChannelID   bool
+	userMemberMap  map[string]*discordgo.Member
+	guildID        string
+	webhookID      string
+	webhookToken   string
+	channelInfoMap map[string]*config.ChannelInfo
 	sync.RWMutex
 }
 
@@ -37,11 +38,10 @@ func New(cfg config.Protocol, account string, c chan config.Message) *bdiscord {
 	b.Remote = c
 	b.Account = account
 	b.userMemberMap = make(map[string]*discordgo.Member)
+	b.channelInfoMap = make(map[string]*config.ChannelInfo)
 	if b.Config.WebhookURL != "" {
 		flog.Debug("Configuring Discord Incoming Webhook")
-		webhookURLSplit := strings.Split(b.Config.WebhookURL, "/")
-		b.webhookToken = webhookURLSplit[len(webhookURLSplit)-1]
-		b.webhookID = webhookURLSplit[len(webhookURLSplit)-2]
+		b.webhookID, b.webhookToken = b.splitURL(b.Config.WebhookURL)
 	}
 	return b
 }
@@ -66,12 +66,13 @@ func (b *bdiscord) Connect() error {
 	b.c.AddHandler(b.messageCreate)
 	b.c.AddHandler(b.memberUpdate)
 	b.c.AddHandler(b.messageUpdate)
+	b.c.AddHandler(b.messageDelete)
 	err = b.c.Open()
 	if err != nil {
 		flog.Debugf("%#v", err)
 		return err
 	}
-	guilds, err := b.c.UserGuilds()
+	guilds, err := b.c.UserGuilds(100, "", "")
 	if err != nil {
 		flog.Debugf("%#v", err)
 		return err
@@ -99,37 +100,75 @@ func (b *bdiscord) Disconnect() error {
 	return nil
 }
 
-func (b *bdiscord) JoinChannel(channel string) error {
-	idcheck := strings.Split(channel, "ID:")
+func (b *bdiscord) JoinChannel(channel config.ChannelInfo) error {
+	b.channelInfoMap[channel.ID] = &channel
+	idcheck := strings.Split(channel.Name, "ID:")
 	if len(idcheck) > 1 {
 		b.UseChannelID = true
 	}
 	return nil
 }
 
-func (b *bdiscord) Send(msg config.Message) error {
+func (b *bdiscord) Send(msg config.Message) (string, error) {
 	flog.Debugf("Receiving %#v", msg)
 	channelID := b.getChannelID(msg.Channel)
 	if channelID == "" {
 		flog.Errorf("Could not find channelID for %v", msg.Channel)
-		return nil
+		return "", nil
 	}
-	if b.Config.WebhookURL == "" {
+	if msg.Event == config.EVENT_USER_ACTION {
+		msg.Text = "_" + msg.Text + "_"
+	}
+
+	wID := b.webhookID
+	wToken := b.webhookToken
+	if ci, ok := b.channelInfoMap[msg.Channel+b.Account]; ok {
+		if ci.Options.WebhookURL != "" {
+			wID, wToken = b.splitURL(ci.Options.WebhookURL)
+		}
+	}
+
+	if wID == "" {
 		flog.Debugf("Broadcasting using token (API)")
-		b.c.ChannelMessageSend(channelID, msg.Username+msg.Text)
-	} else {
-		flog.Debugf("Broadcasting using Webhook")
-		b.c.WebhookExecute(
-			b.webhookID,
-			b.webhookToken,
-			true,
-			&discordgo.WebhookParams{
-				Content:   msg.Text,
-				Username:  msg.Username,
-				AvatarURL: msg.Avatar,
-			})
+		if msg.Event == config.EVENT_MSG_DELETE {
+			if msg.ID == "" {
+				return "", nil
+			}
+			err := b.c.ChannelMessageDelete(channelID, msg.ID)
+			return "", err
+		}
+		if msg.ID != "" {
+			_, err := b.c.ChannelMessageEdit(channelID, msg.ID, msg.Username+msg.Text)
+			return msg.ID, err
+		}
+		res, err := b.c.ChannelMessageSend(channelID, msg.Username+msg.Text)
+		if err != nil {
+			return "", err
+		}
+		return res.ID, err
 	}
-	return nil
+	flog.Debugf("Broadcasting using Webhook")
+	err := b.c.WebhookExecute(
+		wID,
+		wToken,
+		true,
+		&discordgo.WebhookParams{
+			Content:   msg.Text,
+			Username:  msg.Username,
+			AvatarURL: msg.Avatar,
+		})
+	return "", err
+}
+
+func (b *bdiscord) messageDelete(s *discordgo.Session, m *discordgo.MessageDelete) {
+	rmsg := config.Message{Account: b.Account, ID: m.ID, Event: config.EVENT_MSG_DELETE, Text: config.EVENT_MSG_DELETE}
+	rmsg.Channel = b.getChannelName(m.ChannelID)
+	if b.UseChannelID {
+		rmsg.Channel = "ID:" + m.ChannelID
+	}
+	flog.Debugf("Sending message from %s to gateway", b.Account)
+	flog.Debugf("Message is %#v", rmsg)
+	b.Remote <- rmsg
 }
 
 func (b *bdiscord) messageUpdate(s *discordgo.Session, m *discordgo.MessageUpdate) {
@@ -150,40 +189,61 @@ func (b *bdiscord) messageCreate(s *discordgo.Session, m *discordgo.MessageCreat
 		return
 	}
 	// if using webhooks, do not relay if it's ours
-	if b.Config.WebhookURL != "" && m.Author.Bot && m.Author.ID == b.webhookID {
+	if b.useWebhook() && m.Author.Bot && b.isWebhookID(m.Author.ID) {
 		return
 	}
+
 	if len(m.Attachments) > 0 {
 		for _, attach := range m.Attachments {
 			m.Content = m.Content + "\n" + attach.URL
 		}
 	}
-	if m.Content == "" {
-		return
-	}
-	flog.Debugf("Receiving message %#v", m.Message)
-	channelName := b.getChannelName(m.ChannelID)
-	if b.UseChannelID {
-		channelName = "ID:" + m.ChannelID
-	}
-	username := b.getNick(m.Author)
-	if len(m.MentionRoles) > 0 {
-		m.Message.Content = b.replaceRoleMentions(m.Message.Content)
-	}
-	m.Message.Content = b.stripCustomoji(m.Message.Content)
-	m.Message.Content = b.replaceChannelMentions(m.Message.Content)
 
-	text := m.ContentWithMentionsReplaced()
+	var text string
+	if m.Content != "" {
+		flog.Debugf("Receiving message %#v", m.Message)
+		if len(m.MentionRoles) > 0 {
+			m.Message.Content = b.replaceRoleMentions(m.Message.Content)
+		}
+		m.Message.Content = b.stripCustomoji(m.Message.Content)
+		m.Message.Content = b.replaceChannelMentions(m.Message.Content)
+		text = m.ContentWithMentionsReplaced()
+	}
+
+	rmsg := config.Message{Account: b.Account, Avatar: "https://cdn.discordapp.com/avatars/" + m.Author.ID + "/" + m.Author.Avatar + ".jpg",
+		UserID: m.Author.ID, ID: m.ID}
+
+	rmsg.Channel = b.getChannelName(m.ChannelID)
+	if b.UseChannelID {
+		rmsg.Channel = "ID:" + m.ChannelID
+	}
+
+	if !b.Config.UseUserName {
+		rmsg.Username = b.getNick(m.Author)
+	} else {
+		rmsg.Username = m.Author.Username
+	}
+
 	if b.Config.ShowEmbeds && m.Message.Embeds != nil {
 		for _, embed := range m.Message.Embeds {
 			text = text + "embed: " + embed.Title + " - " + embed.Description + " - " + embed.URL + "\n"
 		}
 	}
 
+	// no empty messages
+	if text == "" {
+		return
+	}
+
+	text, ok := b.replaceAction(text)
+	if ok {
+		rmsg.Event = config.EVENT_USER_ACTION
+	}
+
+	rmsg.Text = text
 	flog.Debugf("Sending message from %s on %s to gateway", m.Author.Username, b.Account)
-	b.Remote <- config.Message{Username: username, Text: text, Channel: channelName,
-		Account: b.Account, Avatar: "https://cdn.discordapp.com/avatars/" + m.Author.ID + "/" + m.Author.Avatar + ".jpg",
-		UserID: m.Author.ID}
+	flog.Debugf("Message is %#v", rmsg)
+	b.Remote <- rmsg
 }
 
 func (b *bdiscord) memberUpdate(s *discordgo.Session, m *discordgo.GuildMemberUpdate) {
@@ -275,8 +335,53 @@ func (b *bdiscord) replaceChannelMentions(text string) string {
 	return text
 }
 
+func (b *bdiscord) replaceAction(text string) (string, bool) {
+	if strings.HasPrefix(text, "_") && strings.HasSuffix(text, "_") {
+		return strings.Replace(text, "_", "", -1), true
+	}
+	return text, false
+}
+
 func (b *bdiscord) stripCustomoji(text string) string {
 	// <:doge:302803592035958784>
 	re := regexp.MustCompile("<(:.*?:)[0-9]+>")
 	return re.ReplaceAllString(text, `$1`)
 }
+
+// splitURL splits a webhookURL and returns the id and token
+func (b *bdiscord) splitURL(url string) (string, string) {
+	webhookURLSplit := strings.Split(url, "/")
+	return webhookURLSplit[len(webhookURLSplit)-2], webhookURLSplit[len(webhookURLSplit)-1]
+}
+
+// useWebhook returns true if we have a webhook defined somewhere
+func (b *bdiscord) useWebhook() bool {
+	if b.Config.WebhookURL != "" {
+		return true
+	}
+	for _, channel := range b.channelInfoMap {
+		if channel.Options.WebhookURL != "" {
+			return true
+		}
+	}
+	return false
+}
+
+// isWebhookID returns true if the specified id is used in a defined webhook
+func (b *bdiscord) isWebhookID(id string) bool {
+	if b.Config.WebhookURL != "" {
+		wID, _ := b.splitURL(b.Config.WebhookURL)
+		if wID == id {
+			return true
+		}
+	}
+	for _, channel := range b.channelInfoMap {
+		if channel.Options.WebhookURL != "" {
+			wID, _ := b.splitURL(channel.Options.WebhookURL)
+			if wID == id {
+				return true
+			}
+		}
+	}
+	return false
+}

bridge/gitter/gitter.go

@@ -2,9 +2,9 @@ package bgitter
 
 import (
 	"fmt"
+	"github.com/42wim/go-gitter"
 	"github.com/42wim/matterbridge/bridge/config"
 	log "github.com/Sirupsen/logrus"
-	"github.com/sromku/go-gitter"
 	"strings"
 )
 
@@ -13,6 +13,7 @@ type Bgitter struct {
 	Config  *config.Protocol
 	Remote  chan config.Message
 	Account string
+	User    *gitter.User
 	Users   []gitter.User
 	Rooms   []gitter.Room
 }
@@ -36,7 +37,7 @@ func (b *Bgitter) Connect() error {
 	var err error
 	flog.Info("Connecting")
 	b.c = gitter.New(b.Config.Token)
-	_, err = b.c.GetUser()
+	b.User, err = b.c.GetUser()
 	if err != nil {
 		flog.Debugf("%#v", err)
 		return err
@@ -51,10 +52,10 @@ func (b *Bgitter) Disconnect() error {
 
 }
 
-func (b *Bgitter) JoinChannel(channel string) error {
-	roomID, err := b.c.GetRoomId(channel)
+func (b *Bgitter) JoinChannel(channel config.ChannelInfo) error {
+	roomID, err := b.c.GetRoomId(channel.Name)
 	if err != nil {
-		return fmt.Errorf("Could not find roomID for %v. Please create the room on gitter.im", channel)
+		return fmt.Errorf("Could not find roomID for %v. Please create the room on gitter.im", channel.Name)
 	}
 	room, err := b.c.GetRoom(roomID)
 	if err != nil {
@@ -78,11 +79,17 @@ func (b *Bgitter) JoinChannel(channel string) error {
 		for event := range stream.Event {
 			switch ev := event.Data.(type) {
 			case *gitter.MessageReceived:
-				// check for ZWSP to see if it's not an echo
-				if !strings.HasSuffix(ev.Message.Text, "​") {
+				if ev.Message.From.ID != b.User.ID {
 					flog.Debugf("Sending message from %s on %s to gateway", ev.Message.From.Username, b.Account)
-					b.Remote <- config.Message{Username: ev.Message.From.Username, Text: ev.Message.Text, Channel: room,
-						Account: b.Account, Avatar: b.getAvatar(ev.Message.From.Username), UserID: ev.Message.From.ID}
+					rmsg := config.Message{Username: ev.Message.From.Username, Text: ev.Message.Text, Channel: room,
+						Account: b.Account, Avatar: b.getAvatar(ev.Message.From.Username), UserID: ev.Message.From.ID,
+						ID: ev.Message.ID}
+					if strings.HasPrefix(ev.Message.Text, "@"+ev.Message.From.Username) {
+						rmsg.Event = config.EVENT_USER_ACTION
+						rmsg.Text = strings.Replace(rmsg.Text, "@"+ev.Message.From.Username+" ", "", -1)
+					}
+					flog.Debugf("Message is %#v", rmsg)
+					b.Remote <- rmsg
 				}
 			case *gitter.GitterConnectionClosed:
 				flog.Errorf("connection with gitter closed for room %s", room)
@@ -92,15 +99,37 @@ func (b *Bgitter) JoinChannel(channel string) error {
 	return nil
 }
 
-func (b *Bgitter) Send(msg config.Message) error {
+func (b *Bgitter) Send(msg config.Message) (string, error) {
 	flog.Debugf("Receiving %#v", msg)
 	roomID := b.getRoomID(msg.Channel)
 	if roomID == "" {
 		flog.Errorf("Could not find roomID for %v", msg.Channel)
-		return nil
+		return "", nil
 	}
-	// add ZWSP because gitter echoes our own messages
-	return b.c.SendMessage(roomID, msg.Username+msg.Text+" ​")
+	if msg.Event == config.EVENT_MSG_DELETE {
+		if msg.ID == "" {
+			return "", nil
+		}
+		// gitter has no delete message api
+		_, err := b.c.UpdateMessage(roomID, msg.ID, "")
+		if err != nil {
+			return "", err
+		}
+		return "", nil
+	}
+	if msg.ID != "" {
+		flog.Debugf("updating message with id %s", msg.ID)
+		_, err := b.c.UpdateMessage(roomID, msg.ID, msg.Username+msg.Text)
+		if err != nil {
+			return "", err
+		}
+		return "", nil
+	}
+	resp, err := b.c.SendMessage(roomID, msg.Username+msg.Text)
+	if err != nil {
+		return "", err
+	}
+	return resp.ID, nil
 }
 
 func (b *Bgitter) getRoomID(channel string) string {

bridge/irc/helper.go

@@ -4,6 +4,7 @@ import (
 	"strings"
 )
 
+/*
 func tableformatter(nicks []string, nicksPerRow int, continued bool) string {
 	result := "|IRC users"
 	if continued {
@@ -29,6 +30,7 @@ func tableformatter(nicks []string, nicksPerRow int, continued bool) string {
 	}
 	return result
 }
+*/
 
 func plainformatter(nicks []string, nicksPerRow int) string {
 	return strings.Join(nicks, ", ") + " currently on IRC"

bridge/irc/irc.go

@@ -1,12 +1,18 @@
 package birc
 
 import (
+	"bytes"
 	"crypto/tls"
 	"fmt"
+	"github.com/42wim/go-ircevent"
 	"github.com/42wim/matterbridge/bridge/config"
 	log "github.com/Sirupsen/logrus"
+	"github.com/paulrosania/go-charset/charset"
+	_ "github.com/paulrosania/go-charset/data"
+	"github.com/saintfish/chardet"
 	ircm "github.com/sorcix/irc"
-	"github.com/thoj/go-ircevent"
+	"io"
+	"io/ioutil"
 	"regexp"
 	"sort"
 	"strconv"
@@ -82,6 +88,7 @@ func (b *Birc) Connect() error {
 		i.Password = b.Config.Password
 	}
 	i.AddCallback(ircm.RPL_WELCOME, b.handleNewConnection)
+	i.AddCallback(ircm.RPL_ENDOFMOTD, b.handleOtherAuth)
 	err := i.Connect(b.Config.Server)
 	if err != nil {
 		return err
@@ -112,19 +119,38 @@ func (b *Birc) Disconnect() error {
 	return nil
 }
 
-func (b *Birc) JoinChannel(channel string) error {
-	b.i.Join(channel)
+func (b *Birc) JoinChannel(channel config.ChannelInfo) error {
+	if channel.Options.Key != "" {
+		flog.Debugf("using key %s for channel %s", channel.Options.Key, channel.Name)
+		b.i.Join(channel.Name + " " + channel.Options.Key)
+	} else {
+		b.i.Join(channel.Name)
+	}
 	return nil
 }
 
-func (b *Birc) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
-	if msg.Account == b.Account {
-		return nil
+func (b *Birc) Send(msg config.Message) (string, error) {
+	// ignore delete messages
+	if msg.Event == config.EVENT_MSG_DELETE {
+		return "", nil
 	}
+	flog.Debugf("Receiving %#v", msg)
 	if strings.HasPrefix(msg.Text, "!") {
 		b.Command(&msg)
 	}
+
+	if b.Config.Charset != "" {
+		buf := new(bytes.Buffer)
+		w, err := charset.NewWriter(b.Config.Charset, buf)
+		if err != nil {
+			flog.Errorf("charset from utf-8 conversion failed: %s", err)
+			return "", err
+		}
+		fmt.Fprintf(w, msg.Text)
+		w.Close()
+		msg.Text = buf.String()
+	}
+
 	for _, text := range strings.Split(msg.Text, "\n") {
 		if len(text) > b.Config.MessageLength {
 			text = text[:b.Config.MessageLength] + " <message clipped>"
@@ -133,20 +159,24 @@ func (b *Birc) Send(msg config.Message) error {
 			if len(b.Local) == b.Config.MessageQueue-1 {
 				text = text + " <message clipped>"
 			}
-			b.Local <- config.Message{Text: text, Username: msg.Username, Channel: msg.Channel}
+			b.Local <- config.Message{Text: text, Username: msg.Username, Channel: msg.Channel, Event: msg.Event}
 		} else {
 			flog.Debugf("flooding, dropping message (queue at %d)", len(b.Local))
 		}
 	}
-	return nil
+	return "", nil
 }
 
 func (b *Birc) doSend() {
 	rate := time.Millisecond * time.Duration(b.Config.MessageDelay)
-	throttle := time.Tick(rate)
+	throttle := time.NewTicker(rate)
 	for msg := range b.Local {
-		<-throttle
-		b.i.Privmsg(msg.Channel, msg.Username+msg.Text)
+		<-throttle.C
+		if msg.Event == config.EVENT_USER_ACTION {
+			b.i.Action(msg.Channel, msg.Username+msg.Text)
+		} else {
+			b.i.Privmsg(msg.Channel, msg.Username+msg.Text)
+		}
 	}
 }
 
@@ -228,6 +258,13 @@ func (b *Birc) handleOther(event *irc.Event) {
 	flog.Debugf("%#v", event.Raw)
 }
 
+func (b *Birc) handleOtherAuth(event *irc.Event) {
+	if strings.EqualFold(b.Config.NickServNick, "Q@CServe.quakenet.org") {
+		flog.Debugf("Authenticating %s against %s", b.Config.NickServUsername, b.Config.NickServNick)
+		b.i.Privmsg(b.Config.NickServNick, "AUTH "+b.Config.NickServUsername+" "+b.Config.NickServPassword)
+	}
+}
+
 func (b *Birc) handlePrivMsg(event *irc.Event) {
 	b.Nick = b.i.GetNick()
 	// freenode doesn't send 001 as first reply
@@ -242,17 +279,47 @@ func (b *Birc) handlePrivMsg(event *irc.Event) {
 	if event.Nick == b.Nick {
 		return
 	}
+	rmsg := config.Message{Username: event.Nick, Channel: event.Arguments[0], Account: b.Account, UserID: event.User + "@" + event.Host}
 	flog.Debugf("handlePrivMsg() %s %s %#v", event.Nick, event.Message(), event)
 	msg := ""
 	if event.Code == "CTCP_ACTION" {
-		msg = event.Nick + " "
+		//	msg = event.Nick + " "
+		rmsg.Event = config.EVENT_USER_ACTION
 	}
 	msg += event.Message()
 	// strip IRC colors
 	re := regexp.MustCompile(`[[:cntrl:]](\d+,|)\d+`)
 	msg = re.ReplaceAllString(msg, "")
+
+	var r io.Reader
+	var err error
+	mycharset := b.Config.Charset
+	if mycharset == "" {
+		// detect what were sending so that we convert it to utf-8
+		detector := chardet.NewTextDetector()
+		result, err := detector.DetectBest([]byte(msg))
+		if err != nil {
+			flog.Infof("detection failed for msg: %#v", msg)
+			return
+		}
+		flog.Debugf("detected %s confidence %#v", result.Charset, result.Confidence)
+		mycharset = result.Charset
+		// if we're not sure, just pick ISO-8859-1
+		if result.Confidence < 80 {
+			mycharset = "ISO-8859-1"
+		}
+	}
+	r, err = charset.NewReader(mycharset, strings.NewReader(msg))
+	if err != nil {
+		flog.Errorf("charset to utf-8 conversion failed: %s", err)
+		return
+	}
+	output, _ := ioutil.ReadAll(r)
+	msg = string(output)
+
 	flog.Debugf("Sending message from %s on %s to gateway", event.Arguments[0], b.Account)
-	b.Remote <- config.Message{Username: event.Nick, Text: msg, Channel: event.Arguments[0], Account: b.Account, UserID: event.User + "@" + event.Host}
+	rmsg.Text = msg
+	b.Remote <- rmsg
 }
 
 func (b *Birc) handleTopicWhoTime(event *irc.Event) {

bridge/matrix/matrix.go

@@ -63,23 +63,32 @@ func (b *Bmatrix) Disconnect() error {
 	return nil
 }
 
-func (b *Bmatrix) JoinChannel(channel string) error {
-	resp, err := b.mc.JoinRoom(channel, "", nil)
+func (b *Bmatrix) JoinChannel(channel config.ChannelInfo) error {
+	resp, err := b.mc.JoinRoom(channel.Name, "", nil)
 	if err != nil {
 		return err
 	}
 	b.Lock()
-	b.RoomMap[resp.RoomID] = channel
+	b.RoomMap[resp.RoomID] = channel.Name
 	b.Unlock()
 	return err
 }
 
-func (b *Bmatrix) Send(msg config.Message) error {
+func (b *Bmatrix) Send(msg config.Message) (string, error) {
 	flog.Debugf("Receiving %#v", msg)
+	// ignore delete messages
+	if msg.Event == config.EVENT_MSG_DELETE {
+		return "", nil
+	}
 	channel := b.getRoomID(msg.Channel)
 	flog.Debugf("Sending to channel %s", channel)
+	if msg.Event == config.EVENT_USER_ACTION {
+		b.mc.SendMessageEvent(channel, "m.room.message",
+			matrix.TextMessage{"m.emote", msg.Username + msg.Text})
+		return "", nil
+	}
 	b.mc.SendText(channel, msg.Username+msg.Text)
-	return nil
+	return "", nil
 }
 
 func (b *Bmatrix) getRoomID(channel string) string {
@@ -95,7 +104,7 @@ func (b *Bmatrix) getRoomID(channel string) string {
 func (b *Bmatrix) handlematrix() error {
 	syncer := b.mc.Syncer.(*matrix.DefaultSyncer)
 	syncer.OnEventType("m.room.message", func(ev *matrix.Event) {
-		if ev.Content["msgtype"].(string) == "m.text" && ev.Sender != b.UserID {
+		if (ev.Content["msgtype"].(string) == "m.text" || ev.Content["msgtype"].(string) == "m.notice" || ev.Content["msgtype"].(string) == "m.emote") && ev.Sender != b.UserID {
 			b.RLock()
 			channel, ok := b.RoomMap[ev.RoomID]
 			b.RUnlock()
@@ -108,8 +117,12 @@ func (b *Bmatrix) handlematrix() error {
 				re := regexp.MustCompile("(.*?):.*")
 				username = re.ReplaceAllString(username, `$1`)
 			}
+			rmsg := config.Message{Username: username, Text: ev.Content["body"].(string), Channel: channel, Account: b.Account, UserID: ev.Sender}
+			if ev.Content["msgtype"].(string) == "m.emote" {
+				rmsg.Event = config.EVENT_USER_ACTION
+			}
 			flog.Debugf("Sending message from %s on %s to gateway", ev.Sender, b.Account)
-			b.Remote <- config.Message{Username: username, Text: ev.Content["body"].(string), Channel: channel, Account: b.Account, UserID: ev.Sender}
+			b.Remote <- rmsg
 		}
 		flog.Debugf("Received: %#v", ev)
 	})

bridge/mattermost/mattermost.go

@@ -1,10 +1,13 @@
 package bmattermost
 
 import (
+	"errors"
+	"fmt"
 	"github.com/42wim/matterbridge/bridge/config"
 	"github.com/42wim/matterbridge/matterclient"
 	"github.com/42wim/matterbridge/matterhook"
 	log "github.com/Sirupsen/logrus"
+	"strings"
 )
 
 type MMhook struct {
@@ -12,9 +15,8 @@ type MMhook struct {
 }
 
 type MMapi struct {
-	mc            *matterclient.MMClient
-	mmMap         map[string]string
-	mmIgnoreNicks []string
+	mc    *matterclient.MMClient
+	mmMap map[string]string
 }
 
 type MMMessage struct {
@@ -22,6 +24,9 @@ type MMMessage struct {
 	Channel  string
 	Username string
 	UserID   string
+	ID       string
+	Event    string
+	Extra    map[string][]interface{}
 }
 
 type Bmattermost struct {
@@ -29,7 +34,6 @@ type Bmattermost struct {
 	MMapi
 	Config  *config.Protocol
 	Remote  chan config.Message
-	name    string
 	TeamId  string
 	Account string
 }
@@ -55,33 +59,72 @@ func (b *Bmattermost) Command(cmd string) string {
 }
 
 func (b *Bmattermost) Connect() error {
-	if b.Config.WebhookURL != "" && b.Config.WebhookBindAddress != "" {
-		flog.Info("Connecting using webhookurl and webhookbindaddress")
-		b.mh = matterhook.New(b.Config.WebhookURL,
-			matterhook.Config{InsecureSkipVerify: b.Config.SkipTLSVerify,
-				BindAddress: b.Config.WebhookBindAddress})
-	} else if b.Config.WebhookURL != "" {
-		flog.Info("Connecting using webhookurl (for posting) and token")
+	if b.Config.WebhookBindAddress != "" {
+		if b.Config.WebhookURL != "" {
+			flog.Info("Connecting using webhookurl (sending) and webhookbindaddress (receiving)")
+			b.mh = matterhook.New(b.Config.WebhookURL,
+				matterhook.Config{InsecureSkipVerify: b.Config.SkipTLSVerify,
+					BindAddress: b.Config.WebhookBindAddress})
+		} else if b.Config.Token != "" {
+			flog.Info("Connecting using token (sending)")
+			err := b.apiLogin()
+			if err != nil {
+				return err
+			}
+		} else if b.Config.Login != "" {
+			flog.Info("Connecting using login/password (sending)")
+			err := b.apiLogin()
+			if err != nil {
+				return err
+			}
+		} else {
+			flog.Info("Connecting using webhookbindaddress (receiving)")
+			b.mh = matterhook.New(b.Config.WebhookURL,
+				matterhook.Config{InsecureSkipVerify: b.Config.SkipTLSVerify,
+					BindAddress: b.Config.WebhookBindAddress})
+		}
+		go b.handleMatter()
+		return nil
+	}
+	if b.Config.WebhookURL != "" {
+		flog.Info("Connecting using webhookurl (sending)")
 		b.mh = matterhook.New(b.Config.WebhookURL,
 			matterhook.Config{InsecureSkipVerify: b.Config.SkipTLSVerify,
 				DisableServer: true})
-	} else {
-		flog.Info("Connecting using token")
-		b.mc = matterclient.New(b.Config.Login, b.Config.Password,
-			b.Config.Team, b.Config.Server)
-		b.mc.SkipTLSVerify = b.Config.SkipTLSVerify
-		b.mc.NoTLS = b.Config.NoTLS
-		flog.Infof("Connecting %s (team: %s) on %s", b.Config.Login, b.Config.Team, b.Config.Server)
-		err := b.mc.Login()
+		if b.Config.Token != "" {
+			flog.Info("Connecting using token (receiving)")
+			err := b.apiLogin()
+			if err != nil {
+				return err
+			}
+			go b.handleMatter()
+		} else if b.Config.Login != "" {
+			flog.Info("Connecting using login/password (receiving)")
+			err := b.apiLogin()
+			if err != nil {
+				return err
+			}
+			go b.handleMatter()
+		}
+		return nil
+	} else if b.Config.Token != "" {
+		flog.Info("Connecting using token (sending and receiving)")
+		err := b.apiLogin()
 		if err != nil {
 			return err
 		}
-		flog.Info("Connection succeeded")
-		b.TeamId = b.mc.GetTeamId()
-		go b.mc.WsReceiver()
-		go b.mc.StatusLoop()
+		go b.handleMatter()
+	} else if b.Config.Login != "" {
+		flog.Info("Connecting using login/password (sending and receiving)")
+		err := b.apiLogin()
+		if err != nil {
+			return err
+		}
+		go b.handleMatter()
+	}
+	if b.Config.WebhookBindAddress == "" && b.Config.WebhookURL == "" && b.Config.Login == "" && b.Config.Token == "" {
+		return errors.New("No connection method found. See that you have WebhookBindAddress, WebhookURL or Token/Login/Password/Server/Team configured.")
 	}
-	go b.handleMatter()
 	return nil
 }
 
@@ -89,16 +132,23 @@ func (b *Bmattermost) Disconnect() error {
 	return nil
 }
 
-func (b *Bmattermost) JoinChannel(channel string) error {
+func (b *Bmattermost) JoinChannel(channel config.ChannelInfo) error {
 	// we can only join channels using the API
 	if b.Config.WebhookURL == "" && b.Config.WebhookBindAddress == "" {
-		return b.mc.JoinChannel(b.mc.GetChannelId(channel, ""))
+		id := b.mc.GetChannelId(channel.Name, "")
+		if id == "" {
+			return fmt.Errorf("Could not find channel ID for channel %s", channel.Name)
+		}
+		return b.mc.JoinChannel(id)
 	}
 	return nil
 }
 
-func (b *Bmattermost) Send(msg config.Message) error {
+func (b *Bmattermost) Send(msg config.Message) (string, error) {
 	flog.Debugf("Receiving %#v", msg)
+	if msg.Event == config.EVENT_USER_ACTION {
+		msg.Text = "*" + msg.Text + "*"
+	}
 	nick := msg.Username
 	message := msg.Text
 	channel := msg.Channel
@@ -113,29 +163,71 @@ func (b *Bmattermost) Send(msg config.Message) error {
 		matterMessage.UserName = nick
 		matterMessage.Type = ""
 		matterMessage.Text = message
+		matterMessage.Text = message
+		matterMessage.Props = make(map[string]interface{})
+		matterMessage.Props["matterbridge"] = true
 		err := b.mh.Send(matterMessage)
 		if err != nil {
 			flog.Info(err)
-			return err
+			return "", err
 		}
-		return nil
+		return "", nil
 	}
-	b.mc.PostMessage(b.mc.GetChannelId(channel, ""), message)
-	return nil
+	if msg.Event == config.EVENT_MSG_DELETE {
+		if msg.ID == "" {
+			return "", nil
+		}
+		return msg.ID, b.mc.DeleteMessage(msg.ID)
+	}
+	if msg.Extra != nil {
+		if len(msg.Extra["file"]) > 0 {
+			var err error
+			var res, id string
+			for _, f := range msg.Extra["file"] {
+				fi := f.(config.FileInfo)
+				id, err = b.mc.UploadFile(*fi.Data, b.mc.GetChannelId(channel, ""), fi.Name)
+				if err != nil {
+					flog.Debugf("ERROR %#v", err)
+					return "", err
+				}
+				message = "uploaded a file: " + fi.Name
+				if b.Config.PrefixMessagesWithNick {
+					message = nick + "uploaded a file: " + fi.Name
+				}
+				res, err = b.mc.PostMessageWithFiles(b.mc.GetChannelId(channel, ""), message, []string{id})
+			}
+			return res, err
+		}
+	}
+	if msg.ID != "" {
+		return b.mc.EditMessage(msg.ID, message)
+	}
+	return b.mc.PostMessage(b.mc.GetChannelId(channel, ""), message)
 }
 
 func (b *Bmattermost) handleMatter() {
 	mchan := make(chan *MMMessage)
-	if b.Config.WebhookBindAddress != "" && b.Config.WebhookURL != "" {
+	if b.Config.WebhookBindAddress != "" {
 		flog.Debugf("Choosing webhooks based receiving")
 		go b.handleMatterHook(mchan)
 	} else {
-		flog.Debugf("Choosing login (api) based receiving")
+		if b.Config.Token != "" {
+			flog.Debugf("Choosing token based receiving")
+		} else {
+			flog.Debugf("Choosing login/password based receiving")
+		}
 		go b.handleMatterClient(mchan)
 	}
 	for message := range mchan {
+		rmsg := config.Message{Username: message.Username, Channel: message.Channel, Account: b.Account, UserID: message.UserID, ID: message.ID, Event: message.Event, Extra: message.Extra}
+		text, ok := b.replaceAction(message.Text)
+		if ok {
+			rmsg.Event = config.EVENT_USER_ACTION
+		}
+		rmsg.Text = text
 		flog.Debugf("Sending message from %s on %s to gateway", message.Username, b.Account)
-		b.Remote <- config.Message{Text: message.Text, Username: message.Username, Channel: message.Channel, Account: b.Account, UserID: message.UserID}
+		flog.Debugf("Message is %#v", rmsg)
+		b.Remote <- rmsg
 	}
 }
 
@@ -152,21 +244,44 @@ func (b *Bmattermost) handleMatterClient(mchan chan *MMMessage) {
 		if (message.Raw.Event == "post_edited") && b.Config.EditDisable {
 			continue
 		}
+
+		m := &MMMessage{Extra: make(map[string][]interface{})}
+
+		props := message.Post.Props
+		if props != nil {
+			if _, ok := props["matterbridge"].(bool); ok {
+				flog.Debugf("sent by matterbridge, ignoring")
+				continue
+			}
+			if _, ok := props["override_username"].(string); ok {
+				message.Username = props["override_username"].(string)
+			}
+			if _, ok := props["attachments"].([]interface{}); ok {
+				m.Extra["attachments"] = props["attachments"].([]interface{})
+			}
+		}
 		// do not post our own messages back to irc
 		// only listen to message from our team
-		if (message.Raw.Event == "posted" || message.Raw.Event == "post_edited") &&
+		if (message.Raw.Event == "posted" || message.Raw.Event == "post_edited" || message.Raw.Event == "post_deleted") &&
 			b.mc.User.Username != message.Username && message.Raw.Data["team_id"].(string) == b.TeamId {
+			// if the message has reactions don't repost it (for now, until we can correlate reaction with message)
+			if message.Post.HasReactions {
+				continue
+			}
 			flog.Debugf("Receiving from matterclient %#v", message)
-			m := &MMMessage{}
 			m.UserID = message.UserID
 			m.Username = message.Username
 			m.Channel = message.Channel
 			m.Text = message.Text
+			m.ID = message.Post.Id
 			if message.Raw.Event == "post_edited" && !b.Config.EditDisable {
 				m.Text = message.Text + b.Config.EditSuffix
 			}
+			if message.Raw.Event == "post_deleted" {
+				m.Event = config.EVENT_MSG_DELETE
+			}
 			if len(message.Post.FileIds) > 0 {
-				for _, link := range b.mc.GetPublicLinks(message.Post.FileIds) {
+				for _, link := range b.mc.GetFileLinks(message.Post.FileIds) {
 					m.Text = m.Text + "\n" + link
 				}
 			}
@@ -187,3 +302,32 @@ func (b *Bmattermost) handleMatterHook(mchan chan *MMMessage) {
 		mchan <- m
 	}
 }
+
+func (b *Bmattermost) apiLogin() error {
+	password := b.Config.Password
+	if b.Config.Token != "" {
+		password = "MMAUTHTOKEN=" + b.Config.Token
+	}
+
+	b.mc = matterclient.New(b.Config.Login, password,
+		b.Config.Team, b.Config.Server)
+	b.mc.SkipTLSVerify = b.Config.SkipTLSVerify
+	b.mc.NoTLS = b.Config.NoTLS
+	flog.Infof("Connecting %s (team: %s) on %s", b.Config.Login, b.Config.Team, b.Config.Server)
+	err := b.mc.Login()
+	if err != nil {
+		return err
+	}
+	flog.Info("Connection succeeded")
+	b.TeamId = b.mc.GetTeamId()
+	go b.mc.WsReceiver()
+	go b.mc.StatusLoop()
+	return nil
+}
+
+func (b *Bmattermost) replaceAction(text string) (string, bool) {
+	if strings.HasPrefix(text, "*") && strings.HasSuffix(text, "*") {
+		return strings.Replace(text, "*", "", -1), true
+	}
+	return text, false
+}

bridge/rocketchat/rocketchat.go

@@ -16,7 +16,6 @@ type Brocketchat struct {
 	MMhook
 	Config  *config.Protocol
 	Remote  chan config.Message
-	name    string
 	Account string
 }
 
@@ -54,11 +53,15 @@ func (b *Brocketchat) Disconnect() error {
 
 }
 
-func (b *Brocketchat) JoinChannel(channel string) error {
+func (b *Brocketchat) JoinChannel(channel config.ChannelInfo) error {
 	return nil
 }
 
-func (b *Brocketchat) Send(msg config.Message) error {
+func (b *Brocketchat) Send(msg config.Message) (string, error) {
+	// ignore delete messages
+	if msg.Event == config.EVENT_MSG_DELETE {
+		return "", nil
+	}
 	flog.Debugf("Receiving %#v", msg)
 	matterMessage := matterhook.OMessage{IconURL: b.Config.IconURL}
 	matterMessage.Channel = msg.Channel
@@ -68,9 +71,9 @@ func (b *Brocketchat) Send(msg config.Message) error {
 	err := b.mh.Send(matterMessage)
 	if err != nil {
 		flog.Info(err)
-		return err
+		return "", err
 	}
-	return nil
+	return "", nil
 }
 
 func (b *Brocketchat) handleRocketHook() {

bridge/slack/slack.go

@@ -1,11 +1,16 @@
 package bslack
 
 import (
+	"bytes"
+	"errors"
 	"fmt"
 	"github.com/42wim/matterbridge/bridge/config"
 	"github.com/42wim/matterbridge/matterhook"
 	log "github.com/Sirupsen/logrus"
 	"github.com/nlopes/slack"
+	"html"
+	"io"
+	"net/http"
 	"regexp"
 	"strings"
 	"time"
@@ -52,22 +57,52 @@ func (b *Bslack) Command(cmd string) string {
 }
 
 func (b *Bslack) Connect() error {
-	if b.Config.WebhookURL != "" && b.Config.WebhookBindAddress != "" {
-		flog.Info("Connecting using webhookurl and webhookbindaddress")
+	if b.Config.WebhookBindAddress != "" {
+		if b.Config.WebhookURL != "" {
+			flog.Info("Connecting using webhookurl (sending) and webhookbindaddress (receiving)")
+			b.mh = matterhook.New(b.Config.WebhookURL,
+				matterhook.Config{InsecureSkipVerify: b.Config.SkipTLSVerify,
+					BindAddress: b.Config.WebhookBindAddress})
+		} else if b.Config.Token != "" {
+			flog.Info("Connecting using token (sending)")
+			b.sc = slack.New(b.Config.Token)
+			b.rtm = b.sc.NewRTM()
+			go b.rtm.ManageConnection()
+			flog.Info("Connecting using webhookbindaddress (receiving)")
+			b.mh = matterhook.New(b.Config.WebhookURL,
+				matterhook.Config{InsecureSkipVerify: b.Config.SkipTLSVerify,
+					BindAddress: b.Config.WebhookBindAddress})
+		} else {
+			flog.Info("Connecting using webhookbindaddress (receiving)")
+			b.mh = matterhook.New(b.Config.WebhookURL,
+				matterhook.Config{InsecureSkipVerify: b.Config.SkipTLSVerify,
+					BindAddress: b.Config.WebhookBindAddress})
+		}
+		go b.handleSlack()
+		return nil
+	}
+	if b.Config.WebhookURL != "" {
+		flog.Info("Connecting using webhookurl (sending)")
 		b.mh = matterhook.New(b.Config.WebhookURL,
-			matterhook.Config{BindAddress: b.Config.WebhookBindAddress})
-	} else if b.Config.WebhookURL != "" {
-		flog.Info("Connecting using webhookurl (for posting) and token")
-		b.mh = matterhook.New(b.Config.WebhookURL,
-			matterhook.Config{DisableServer: true})
-	} else {
-		flog.Info("Connecting using token")
+			matterhook.Config{InsecureSkipVerify: b.Config.SkipTLSVerify,
+				DisableServer: true})
+		if b.Config.Token != "" {
+			flog.Info("Connecting using token (receiving)")
+			b.sc = slack.New(b.Config.Token)
+			b.rtm = b.sc.NewRTM()
+			go b.rtm.ManageConnection()
+			go b.handleSlack()
+		}
+	} else if b.Config.Token != "" {
+		flog.Info("Connecting using token (sending and receiving)")
 		b.sc = slack.New(b.Config.Token)
 		b.rtm = b.sc.NewRTM()
 		go b.rtm.ManageConnection()
+		go b.handleSlack()
+	}
+	if b.Config.WebhookBindAddress == "" && b.Config.WebhookURL == "" && b.Config.Token == "" {
+		return errors.New("No connection method found. See that you have WebhookBindAddress, WebhookURL or Token configured.")
 	}
-	flog.Info("Connection succeeded")
-	go b.handleSlack()
 	return nil
 }
 
@@ -76,14 +111,14 @@ func (b *Bslack) Disconnect() error {
 
 }
 
-func (b *Bslack) JoinChannel(channel string) error {
+func (b *Bslack) JoinChannel(channel config.ChannelInfo) error {
 	// we can only join channels using the API
-	if b.Config.WebhookURL == "" || b.Config.WebhookBindAddress == "" {
+	if b.Config.WebhookURL == "" && b.Config.WebhookBindAddress == "" {
 		if strings.HasPrefix(b.Config.Token, "xoxb") {
 			// TODO check if bot has already joined channel
 			return nil
 		}
-		_, err := b.sc.JoinChannel(channel)
+		_, err := b.sc.JoinChannel(channel.Name)
 		if err != nil {
 			if err.Error() != "name_taken" {
 				return err
@@ -93,8 +128,11 @@ func (b *Bslack) JoinChannel(channel string) error {
 	return nil
 }
 
-func (b *Bslack) Send(msg config.Message) error {
+func (b *Bslack) Send(msg config.Message) (string, error) {
 	flog.Debugf("Receiving %#v", msg)
+	if msg.Event == config.EVENT_USER_ACTION {
+		msg.Text = "_" + msg.Text + "_"
+	}
 	nick := msg.Username
 	message := msg.Text
 	channel := msg.Channel
@@ -110,16 +148,16 @@ func (b *Bslack) Send(msg config.Message) error {
 		err := b.mh.Send(matterMessage)
 		if err != nil {
 			flog.Info(err)
-			return err
+			return "", err
 		}
-		return nil
+		return "", nil
 	}
 	schannel, err := b.getChannelByName(channel)
 	if err != nil {
-		return err
+		return "", err
 	}
 	np := slack.NewPostMessageParameters()
-	if b.Config.PrefixMessagesWithNick == true {
+	if b.Config.PrefixMessagesWithNick {
 		np.AsUser = true
 	}
 	np.Username = nick
@@ -127,14 +165,33 @@ func (b *Bslack) Send(msg config.Message) error {
 	if msg.Avatar != "" {
 		np.IconURL = msg.Avatar
 	}
-	b.sc.PostMessage(schannel.ID, message, np)
+	np.Attachments = append(np.Attachments, slack.Attachment{CallbackID: "matterbridge"})
+	np.Attachments = append(np.Attachments, b.createAttach(msg.Extra)...)
 
-	/*
-	   newmsg := b.rtm.NewOutgoingMessage(message, schannel.ID)
-	   b.rtm.SendMessage(newmsg)
-	*/
+	// replace mentions
+	np.LinkNames = 1
 
-	return nil
+	if msg.Event == config.EVENT_MSG_DELETE {
+		// some protocols echo deletes, but with empty ID
+		if msg.ID == "" {
+			return "", nil
+		}
+		// we get a "slack <ID>", split it
+		ts := strings.Fields(msg.ID)
+		b.sc.DeleteMessage(schannel.ID, ts[1])
+		return "", nil
+	}
+	// if we have no ID it means we're creating a new message, not updating an existing one
+	if msg.ID != "" {
+		ts := strings.Fields(msg.ID)
+		b.sc.UpdateMessage(schannel.ID, ts[1], message)
+		return "", nil
+	}
+	_, id, err := b.sc.PostMessage(schannel.ID, message, np)
+	if err != nil {
+		return "", err
+	}
+	return "slack " + id, nil
 }
 
 func (b *Bslack) getAvatar(user string) string {
@@ -175,7 +232,7 @@ func (b *Bslack) getChannelByID(ID string) (*slack.Channel, error) {
 
 func (b *Bslack) handleSlack() {
 	mchan := make(chan *MMMessage)
-	if b.Config.WebhookBindAddress != "" && b.Config.WebhookURL != "" {
+	if b.Config.WebhookBindAddress != "" {
 		flog.Debugf("Choosing webhooks based receiving")
 		go b.handleMatterHook(mchan)
 	} else {
@@ -189,47 +246,112 @@ func (b *Bslack) handleSlack() {
 		if b.Config.WebhookURL == "" && b.Config.WebhookBindAddress == "" && message.Username == b.si.User.Name {
 			continue
 		}
-		texts := strings.Split(message.Text, "\n")
-		for _, text := range texts {
-			text = b.replaceURL(text)
-			flog.Debugf("Sending message from %s on %s to gateway", message.Username, b.Account)
-			b.Remote <- config.Message{Text: text, Username: message.Username, Channel: message.Channel, Account: b.Account, Avatar: b.getAvatar(message.Username), UserID: message.UserID}
+		if (message.Text == "" || message.Username == "") && message.Raw.SubType != "message_deleted" {
+			continue
 		}
+		text := message.Text
+		text = b.replaceURL(text)
+		text = html.UnescapeString(text)
+		flog.Debugf("Sending message from %s on %s to gateway", message.Username, b.Account)
+		msg := config.Message{Text: text, Username: message.Username, Channel: message.Channel, Account: b.Account, Avatar: b.getAvatar(message.Username), UserID: message.UserID, ID: "slack " + message.Raw.Timestamp, Extra: make(map[string][]interface{})}
+		if message.Raw.SubType == "me_message" {
+			msg.Event = config.EVENT_USER_ACTION
+		}
+		if message.Raw.SubType == "channel_leave" || message.Raw.SubType == "channel_join" {
+			msg.Username = "system"
+			msg.Event = config.EVENT_JOIN_LEAVE
+		}
+		// edited messages have a submessage, use this timestamp
+		if message.Raw.SubMessage != nil {
+			msg.ID = "slack " + message.Raw.SubMessage.Timestamp
+		}
+		if message.Raw.SubType == "message_deleted" {
+			msg.Text = config.EVENT_MSG_DELETE
+			msg.Event = config.EVENT_MSG_DELETE
+			msg.ID = "slack " + message.Raw.DeletedTimestamp
+		}
+
+		// if we have a file attached, download it (in memory) and put a pointer to it in msg.Extra
+		if message.Raw.File != nil {
+			// limit to 1MB for now
+			if message.Raw.File.Size <= 1000000 {
+				data, err := b.downloadFile(message.Raw.File.URLPrivateDownload)
+				if err != nil {
+					flog.Errorf("download %s failed %#v", message.Raw.File.URLPrivateDownload, err)
+				} else {
+					msg.Extra["file"] = append(msg.Extra["file"], config.FileInfo{Name: message.Raw.File.Name, Data: data})
+				}
+			}
+		}
+		flog.Debugf("Message is %#v", msg)
+		b.Remote <- msg
 	}
 }
 
 func (b *Bslack) handleSlackClient(mchan chan *MMMessage) {
-	count := 0
 	for msg := range b.rtm.IncomingEvents {
 		switch ev := msg.Data.(type) {
 		case *slack.MessageEvent:
-			// ignore first message
-			if count > 0 {
-				flog.Debugf("Receiving from slackclient %#v", ev)
-				if !b.Config.EditDisable && ev.SubMessage != nil {
-					flog.Debugf("SubMessage %#v", ev.SubMessage)
-					ev.User = ev.SubMessage.User
-					ev.Text = ev.SubMessage.Text + b.Config.EditSuffix
-				}
-				// use our own func because rtm.GetChannelInfo doesn't work for private channels
-				channel, err := b.getChannelByID(ev.Channel)
-				if err != nil {
+			flog.Debugf("Receiving from slackclient %#v", ev)
+			if len(ev.Attachments) > 0 {
+				// skip messages we made ourselves
+				if ev.Attachments[0].CallbackID == "matterbridge" {
 					continue
 				}
+			}
+			if !b.Config.EditDisable && ev.SubMessage != nil && ev.SubMessage.ThreadTimestamp != ev.SubMessage.Timestamp {
+				flog.Debugf("SubMessage %#v", ev.SubMessage)
+				ev.User = ev.SubMessage.User
+				ev.Text = ev.SubMessage.Text + b.Config.EditSuffix
+
+				// it seems ev.SubMessage.Edited == nil when slack unfurls
+				// do not forward these messages #266
+				if ev.SubMessage.Edited == nil {
+					continue
+				}
+			}
+			// use our own func because rtm.GetChannelInfo doesn't work for private channels
+			channel, err := b.getChannelByID(ev.Channel)
+			if err != nil {
+				continue
+			}
+			m := &MMMessage{}
+			if ev.BotID == "" && ev.SubType != "message_deleted" {
 				user, err := b.rtm.GetUserInfo(ev.User)
 				if err != nil {
 					continue
 				}
-				m := &MMMessage{}
 				m.UserID = user.ID
 				m.Username = user.Name
-				m.Channel = channel.Name
-				m.Text = ev.Text
-				m.Raw = ev
-				m.Text = b.replaceMention(m.Text)
-				mchan <- m
 			}
-			count++
+			m.Channel = channel.Name
+			m.Text = ev.Text
+			if m.Text == "" {
+				for _, attach := range ev.Attachments {
+					if attach.Text != "" {
+						m.Text = attach.Text
+					} else {
+						m.Text = attach.Fallback
+					}
+				}
+			}
+			m.Raw = ev
+			m.Text = b.replaceMention(m.Text)
+			// when using webhookURL we can't check if it's our webhook or not for now
+			if ev.BotID != "" && b.Config.WebhookURL == "" {
+				bot, err := b.rtm.GetBotInfo(ev.BotID)
+				if err != nil {
+					continue
+				}
+				if bot.Name != "" {
+					m.Username = bot.Name
+					if ev.Username != "" {
+						m.Username = ev.Username
+					}
+					m.UserID = bot.ID
+				}
+			}
+			mchan <- m
 		case *slack.OutgoingErrorEvent:
 			flog.Debugf("%#v", ev.Error())
 		case *slack.ChannelJoinedEvent:
@@ -294,3 +416,47 @@ func (b *Bslack) replaceURL(text string) string {
 	}
 	return text
 }
+
+func (b *Bslack) createAttach(extra map[string][]interface{}) []slack.Attachment {
+	var attachs []slack.Attachment
+	for _, v := range extra["attachments"] {
+		entry := v.(map[string]interface{})
+		s := slack.Attachment{}
+		s.Fallback = entry["fallback"].(string)
+		s.Color = entry["color"].(string)
+		s.Pretext = entry["pretext"].(string)
+		s.AuthorName = entry["author_name"].(string)
+		s.AuthorLink = entry["author_link"].(string)
+		s.AuthorIcon = entry["author_icon"].(string)
+		s.Title = entry["title"].(string)
+		s.TitleLink = entry["title_link"].(string)
+		s.Text = entry["text"].(string)
+		s.ImageURL = entry["image_url"].(string)
+		s.ThumbURL = entry["thumb_url"].(string)
+		s.Footer = entry["footer"].(string)
+		s.FooterIcon = entry["footer_icon"].(string)
+		attachs = append(attachs, s)
+	}
+	return attachs
+}
+
+func (b *Bslack) downloadFile(url string) (*[]byte, error) {
+	var buf bytes.Buffer
+	client := &http.Client{
+		Timeout: time.Second * 5,
+	}
+	req, err := http.NewRequest("GET", url, nil)
+	if err != nil {
+		return nil, err
+	}
+	req.Header.Add("Authorization", "Bearer "+b.Config.Token)
+	resp, err := client.Do(req)
+	if err != nil {
+		resp.Body.Close()
+		return nil, err
+	}
+	io.Copy(&buf, resp.Body)
+	data := buf.Bytes()
+	resp.Body.Close()
+	return &data, nil
+}

bridge/steam/steam.go

@@ -60,8 +60,8 @@ func (b *Bsteam) Disconnect() error {
 
 }
 
-func (b *Bsteam) JoinChannel(channel string) error {
-	id, err := steamid.NewId(channel)
+func (b *Bsteam) JoinChannel(channel config.ChannelInfo) error {
+	id, err := steamid.NewId(channel.Name)
 	if err != nil {
 		return err
 	}
@@ -69,13 +69,17 @@ func (b *Bsteam) JoinChannel(channel string) error {
 	return nil
 }
 
-func (b *Bsteam) Send(msg config.Message) error {
+func (b *Bsteam) Send(msg config.Message) (string, error) {
+	// ignore delete messages
+	if msg.Event == config.EVENT_MSG_DELETE {
+		return "", nil
+	}
 	id, err := steamid.NewId(msg.Channel)
 	if err != nil {
-		return err
+		return "", err
 	}
 	b.c.Social.SendMessage(id, steamlang.EChatEntryType_ChatMsg, msg.Username+msg.Text)
-	return nil
+	return "", nil
 }
 
 func (b *Bsteam) getNick(id steamid.SteamId) string {
@@ -101,7 +105,9 @@ func (b *Bsteam) handleEvents() {
 		case *steam.ChatMsgEvent:
 			flog.Debugf("Receiving ChatMsgEvent: %#v", e)
 			flog.Debugf("Sending message from %s on %s to gateway", b.getNick(e.ChatterId), b.Account)
-			msg := config.Message{Username: b.getNick(e.ChatterId), Text: e.Message, Channel: strconv.FormatInt(int64(e.ChatRoomId), 10), Account: b.Account, UserID: strconv.FormatInt(int64(e.ChatterId), 10)}
+			// for some reason we have to remove 0x18000000000000
+			channel := int64(e.ChatRoomId) - 0x18000000000000
+			msg := config.Message{Username: b.getNick(e.ChatterId), Text: e.Message, Channel: strconv.FormatInt(channel, 10), Account: b.Account, UserID: strconv.FormatInt(int64(e.ChatterId), 10)}
 			b.Remote <- msg
 		case *steam.PersonaStateEvent:
 			flog.Debugf("PersonaStateEvent: %#v\n", e)
@@ -134,7 +140,7 @@ func (b *Bsteam) handleEvents() {
 					myLoginInfo.AuthCode = code
 				}
 			default:
-				log.Errorf("LogOnFailedEvent: ", e.Result)
+				log.Errorf("LogOnFailedEvent: %#v ", e.Result)
 				// TODO: Handle EResult_InvalidLoginAuthCode
 				return
 			}

bridge/telegram/telegram.go

@@ -53,30 +53,62 @@ func (b *Btelegram) Disconnect() error {
 
 }
 
-func (b *Btelegram) JoinChannel(channel string) error {
+func (b *Btelegram) JoinChannel(channel config.ChannelInfo) error {
 	return nil
 }
 
-func (b *Btelegram) Send(msg config.Message) error {
+func (b *Btelegram) Send(msg config.Message) (string, error) {
 	flog.Debugf("Receiving %#v", msg)
 	chatid, err := strconv.ParseInt(msg.Channel, 10, 64)
 	if err != nil {
-		return err
+		return "", err
 	}
 
 	if b.Config.MessageFormat == "HTML" {
 		msg.Text = makeHTML(msg.Text)
 	}
+
+	if msg.Event == config.EVENT_MSG_DELETE {
+		if msg.ID == "" {
+			return "", nil
+		}
+		msgid, err := strconv.Atoi(msg.ID)
+		if err != nil {
+			return "", err
+		}
+		_, err = b.c.DeleteMessage(tgbotapi.DeleteMessageConfig{ChatID: chatid, MessageID: msgid})
+		return "", err
+	}
+
+	// edit the message if we have a msg ID
+	if msg.ID != "" {
+		msgid, err := strconv.Atoi(msg.ID)
+		if err != nil {
+			return "", err
+		}
+		m := tgbotapi.NewEditMessageText(chatid, msgid, msg.Username+msg.Text)
+		_, err = b.c.Send(m)
+		if err != nil {
+			return "", err
+		}
+		return "", nil
+	}
+
 	m := tgbotapi.NewMessage(chatid, msg.Username+msg.Text)
 	if b.Config.MessageFormat == "HTML" {
 		m.ParseMode = tgbotapi.ModeHTML
 	}
-	_, err = b.c.Send(m)
-	return err
+	res, err := b.c.Send(m)
+	if err != nil {
+		return "", err
+	}
+	return strconv.Itoa(res.MessageID), nil
+
 }
 
 func (b *Btelegram) handleRecv(updates <-chan tgbotapi.Update) {
 	for update := range updates {
+		flog.Debugf("Receiving from telegram: %#v", update.Message)
 		var message *tgbotapi.Message
 		username := ""
 		channel := ""
@@ -114,23 +146,46 @@ func (b *Btelegram) handleRecv(updates <-chan tgbotapi.Update) {
 		if username == "" {
 			username = "unknown"
 		}
-		if message.Sticker != nil {
+		if message.Sticker != nil && b.Config.UseInsecureURL {
 			text = text + " " + b.getFileDirectURL(message.Sticker.FileID)
 		}
-		if message.Video != nil {
+		if message.Video != nil && b.Config.UseInsecureURL {
 			text = text + " " + b.getFileDirectURL(message.Video.FileID)
 		}
-		if message.Photo != nil {
+		if message.Photo != nil && b.Config.UseInsecureURL {
 			photos := *message.Photo
 			// last photo is the biggest
 			text = text + " " + b.getFileDirectURL(photos[len(photos)-1].FileID)
 		}
-		if message.Document != nil {
+		if message.Document != nil && b.Config.UseInsecureURL {
 			text = text + " " + message.Document.FileName + " : " + b.getFileDirectURL(message.Document.FileID)
 		}
+
+		// quote the previous message
+		if message.ReplyToMessage != nil {
+			usernameReply := ""
+			if message.ReplyToMessage.From != nil {
+				if b.Config.UseFirstName {
+					usernameReply = message.ReplyToMessage.From.FirstName
+				}
+				if usernameReply == "" {
+					usernameReply = message.ReplyToMessage.From.UserName
+					if usernameReply == "" {
+						usernameReply = message.ReplyToMessage.From.FirstName
+					}
+				}
+			}
+			if usernameReply == "" {
+				usernameReply = "unknown"
+			}
+			text = text + " (re @" + usernameReply + ":" + message.ReplyToMessage.Text + ")"
+		}
+
 		if text != "" {
 			flog.Debugf("Sending message from %s on %s to gateway", username, b.Account)
-			b.Remote <- config.Message{Username: username, Text: text, Channel: channel, Account: b.Account, UserID: strconv.Itoa(message.From.ID)}
+			msg := config.Message{Username: username, Text: text, Channel: channel, Account: b.Account, UserID: strconv.Itoa(message.From.ID), ID: strconv.Itoa(message.MessageID)}
+			flog.Debugf("Message is %#v", msg)
+			b.Remote <- msg
 		}
 	}
 }

bridge/xmpp/xmpp.go

@@ -4,6 +4,7 @@ import (
 	"crypto/tls"
 	"github.com/42wim/matterbridge/bridge/config"
 	log "github.com/Sirupsen/logrus"
+	"github.com/jpillora/backoff"
 	"github.com/mattn/go-xmpp"
 
 	"strings"
@@ -43,7 +44,29 @@ func (b *Bxmpp) Connect() error {
 		return err
 	}
 	flog.Info("Connection succeeded")
-	go b.handleXmpp()
+	go func() {
+		initial := true
+		bf := &backoff.Backoff{
+			Min:    time.Second,
+			Max:    5 * time.Minute,
+			Jitter: true,
+		}
+		for {
+			if initial {
+				b.handleXmpp()
+				initial = false
+			}
+			d := bf.Duration()
+			flog.Infof("Disconnected. Reconnecting in %s", d)
+			time.Sleep(d)
+			b.xc, err = b.createXMPP()
+			if err == nil {
+				b.Remote <- config.Message{Username: "system", Text: "rejoin", Channel: "", Account: b.Account, Event: config.EVENT_REJOIN_CHANNELS}
+				b.handleXmpp()
+				bf.Reset()
+			}
+		}
+	}()
 	return nil
 }
 
@@ -51,15 +74,19 @@ func (b *Bxmpp) Disconnect() error {
 	return nil
 }
 
-func (b *Bxmpp) JoinChannel(channel string) error {
-	b.xc.JoinMUCNoHistory(channel+"@"+b.Config.Muc, b.Config.Nick)
+func (b *Bxmpp) JoinChannel(channel config.ChannelInfo) error {
+	b.xc.JoinMUCNoHistory(channel.Name+"@"+b.Config.Muc, b.Config.Nick)
 	return nil
 }
 
-func (b *Bxmpp) Send(msg config.Message) error {
+func (b *Bxmpp) Send(msg config.Message) (string, error) {
+	// ignore delete messages
+	if msg.Event == config.EVENT_MSG_DELETE {
+		return "", nil
+	}
 	flog.Debugf("Receiving %#v", msg)
 	b.xc.Send(xmpp.Chat{Type: "groupchat", Remote: msg.Channel + "@" + b.Config.Muc, Text: msg.Username + msg.Text})
-	return nil
+	return "", nil
 }
 
 func (b *Bxmpp) createXMPP() (*xmpp.Client, error) {
@@ -96,7 +123,11 @@ func (b *Bxmpp) xmppKeepAlive() chan bool {
 		for {
 			select {
 			case <-ticker.C:
-				b.xc.PingC2S("", "")
+				flog.Debugf("PING")
+				err := b.xc.PingC2S("", "")
+				if err != nil {
+					flog.Debugf("PING failed %#v", err)
+				}
 			case <-done:
 				return
 			}
@@ -106,6 +137,7 @@ func (b *Bxmpp) xmppKeepAlive() chan bool {
 }
 
 func (b *Bxmpp) handleXmpp() error {
+	var ok bool
 	done := b.xmppKeepAlive()
 	defer close(done)
 	nodelay := time.Time{}
@@ -127,8 +159,13 @@ func (b *Bxmpp) handleXmpp() error {
 					nick = s[1]
 				}
 				if nick != b.Config.Nick && v.Stamp == nodelay && v.Text != "" {
+					rmsg := config.Message{Username: nick, Text: v.Text, Channel: channel, Account: b.Account, UserID: v.Remote}
+					rmsg.Text, ok = b.replaceAction(rmsg.Text)
+					if ok {
+						rmsg.Event = config.EVENT_USER_ACTION
+					}
 					flog.Debugf("Sending message from %s on %s to gateway", nick, b.Account)
-					b.Remote <- config.Message{Username: nick, Text: v.Text, Channel: channel, Account: b.Account, UserID: v.Remote}
+					b.Remote <- rmsg
 				}
 			}
 		case xmpp.Presence:
@@ -136,3 +173,10 @@ func (b *Bxmpp) handleXmpp() error {
 		}
 	}
 }
+
+func (b *Bxmpp) replaceAction(text string) (string, bool) {
+	if strings.HasPrefix(text, "/me ") {
+		return strings.Replace(text, "/me ", "", -1), true
+	}
+	return text, false
+}

changelog.md

@@ -1,3 +1,187 @@
+# v1.3.1
+## New features
+* Support mattermost 4.3.0 and every other 4.x as api4 should be stable (mattermost)
+## Bugfix
+* Use bot username if specified (slack). Closes #273
+
+# v1.3.0
+## New features
+* Relay slack_attachments from mattermost to slack (slack). Closes #260
+* Add support for quoting previous message when replying (telegram). #237
+* Add support for Quakenet auth (irc). Closes #263
+* Download files (max size 1MB) from slack and reupload to mattermost (slack/mattermost). Closes #255
+
+## Enhancements
+* Backoff for 60 seconds when reconnecting too fast (irc) #267
+* Use override username if specified (mattermost). #260
+
+## Bugfix
+* Try to not forward slack unfurls. Closes #266
+
+# v1.2.0
+## Breaking changes
+* If you're running a discord bridge, update to this release before 16 october otherwise
+it will stop working. (see https://discordapp.com/developers/docs/reference)
+
+## New features
+* general: Add delete support. (actually delete the messages on bridges that support it)
+    (mattermost,discord,gitter,slack,telegram)
+
+## Bugfix
+* Do not break messages on newline (slack). Closes #258 
+* Update telegram library
+* Update discord library (supports v6 API now). Old API is deprecated on 16 October
+
+# v1.1.2
+## New features
+* general: also build darwin binaries
+* mattermost: add support for mattermost 4.2.x
+
+## Bugfix 
+* mattermost: Send images when text is empty regression. (mattermost). Closes #254
+* slack: also send the first messsage after connect. #252
+
+# v1.1.1
+## Bugfix
+* mattermost: fix public links
+
+# v1.1.0
+## New features
+* general: Add better editing support. (actually edit the messages on bridges that support it)
+	(mattermost,discord,gitter,slack,telegram)
+* mattermost: use API v4 (removes support for mattermost < 3.8)
+* mattermost: add support for personal access tokens (since mattermost 4.1)
+	Use ```Token="yourtoken"``` in mattermost config
+	See https://docs.mattermost.com/developer/personal-access-tokens.html for more info
+* matrix: Relay notices (matrix). Closes #243
+* irc: Add a charset option. Closes #247
+
+## Bugfix
+* slack: Handle leave/join events (slack). Closes #246
+* slack: Replace mentions from other bridges. (slack). Closes #233
+* gitter: remove ZWSP after messages
+
+# v1.0.1
+## New features
+* mattermost: add support for mattermost 4.1.x
+* discord: allow a webhookURL per channel #239
+
+# v1.0.0
+## New features
+* general: Add action support for slack,mattermost,irc,gitter,matrix,xmpp,discord. #199
+* discord: Shows the username instead of the server nickname #234
+
+# v1.0.0-rc1
+## New features
+* general: Add action support for slack,mattermost,irc,gitter,matrix,xmpp,discord. #199
+
+## Bugfix
+* general: Handle same account in multiple gateways better
+* mattermost: ignore edited messages with reactions
+* mattermost: Fix double posting of edited messages by using lru cache
+* irc: update vendor
+
+# v0.16.3
+## Bugfix
+* general: Fix in/out logic. Closes #224 
+* general: Fix message modification
+* slack: Disable message from other bots when using webhooks (slack)
+* mattermost: Return better error messages on mattermost connect
+
+# v0.16.2
+## New features
+* general: binary builds against latest commit are now available on https://bintray.com/42wim/nightly/Matterbridge/_latestVersion
+
+## Bugfix
+* slack: fix loop introduced by relaying message of other bots #219
+* slack: Suppress parent message when child message is received #218
+* mattermost: fix regression when using webhookurl and webhookbindaddress #221
+
+# v0.16.1
+## New features
+* slack: also relay messages of other bots #213
+* mattermost: show also links if public links have not been enabled.
+
+## Bugfix
+* mattermost, slack: fix connecting logic #216
+
+# v0.16.0
+## Breaking Changes
+* URL,UseAPI,BindAddress is deprecated. Your config has to be updated.
+  * URL => WebhookURL
+  * BindAddress => WebhookBindAddress
+  * UseAPI => removed 
+  This change allows you to specify a WebhookURL and a token (slack,discord), so that
+  messages will be sent with the webhook, but received via the token (API)
+  If you have not specified WebhookURL and WebhookBindAddress the API (login or token) 
+  will be used automatically. (no need for UseAPI)
+
+## New features
+* mattermost: add support for mattermost 4.0
+* steam: New protocol support added (http://store.steampowered.com/)
+* discord: Support for embedded messages (sent by other bots)
+  Shows title, description and URL of embedded messages (sent by other bots)
+  To enable add ```ShowEmbeds=true``` to your discord config 
+* discord: ```WebhookURL``` posting support added (thanks @saury07) #204
+  Discord API does not allow to change the name of the user posting, but webhooks does.
+
+## Changes
+* general: all :emoji: will be converted to unicode, providing consistent emojis across all bridges
+* telegram: Add ```UseInsecureURL``` option for telegram (default false)
+  WARNING! If enabled this will relay GIF/stickers/documents and other attachments as URLs
+  Those URLs will contain your bot-token. This may not be what you want.
+  For now there is no secure way to relay GIF/stickers/documents without seeing your token.
+
+## Bugfix
+* irc: detect charset and try to convert it to utf-8 before sending it to other bridges. #209 #210
+* slack: Remove label from URLs (slack). #205
+* slack: Relay <>& correctly to other bridges #215
+* steam: Fix channel id bug in steam (channels are off by 0x18000000000000)
+* general: various improvements
+* general: samechannelgateway now relays messages correct again #207
+
+
+# v0.16.0-rc2
+## Breaking Changes
+* URL,UseAPI,BindAddress is deprecated. Your config has to be updated.
+  * URL => WebhookURL
+  * BindAddress => WebhookBindAddress
+  * UseAPI => removed 
+  This change allows you to specify a WebhookURL and a token (slack,discord), so that
+  messages will be sent with the webhook, but received via the token (API)
+  If you have not specified WebhookURL and WebhookBindAddress the API (login or token) 
+  will be used automatically. (no need for UseAPI)
+
+## Bugfix since rc1
+* steam: Fix channel id bug in steam (channels are off by 0x18000000000000)
+* telegram: Add UseInsecureURL option for telegram (default false)
+  WARNING! If enabled this will relay GIF/stickers/documents and other attachments as URLs
+  Those URLs will contain your bot-token. This may not be what you want.
+  For now there is no secure way to relay GIF/stickers/documents without seeing your token.
+* irc: detect charset and try to convert it to utf-8 before sending it to other bridges. #209 #210
+* general: various improvements
+
+
+# v0.16.0-rc1
+## Breaking Changes
+* URL,UseAPI,BindAddress is deprecated. Your config has to be updated.
+  * URL => WebhookURL
+  * BindAddress => WebhookBindAddress
+  * UseAPI => removed 
+  This change allows you to specify a WebhookURL and a token (slack,discord), so that
+  messages will be sent with the webhook, but received via the token (API)
+  If you have not specified WebhookURL and WebhookBindAddress the API (login or token) 
+  will be used automatically. (no need for UseAPI)
+
+## New features
+* steam: New protocol support added (http://store.steampowered.com/)
+* discord: WebhookURL posting support added (thanks @saury07) #204
+  Discord API does not allow to change the name of the user posting, but webhooks does.
+
+## Bugfix
+* general: samechannelgateway now relays messages correct again #207
+* slack: Remove label from URLs (slack). #205
+
 # v0.15.0
 ## New features
 * general: add option IgnoreMessages for all protocols (see mattebridge.toml.sample)

ci/bintray.sh

@@ -0,0 +1,27 @@
+#!/bin/bash
+go version |grep go1.9 || exit
+VERSION=$(git describe --tags)
+mkdir ci/binaries
+GOOS=windows GOARCH=amd64 go build -ldflags "-s -w -X main.githash=$(git log --pretty=format:'%h' -n 1)" -o ci/binaries/matterbridge-$VERSION-windows-amd64.exe
+GOOS=linux GOARCH=amd64 go build -ldflags "-s -w -X main.githash=$(git log --pretty=format:'%h' -n 1)" -o ci/binaries/matterbridge-$VERSION-linux-amd64
+GOOS=linux GOARCH=arm go build -ldflags "-s -w -X main.githash=$(git log --pretty=format:'%h' -n 1)" -o ci/binaries/matterbridge-$VERSION-linux-arm
+GOOS=darwin GOARCH=amd64 go build -ldflags "-s -w -X main.githash=$(git log --pretty=format:'%h' -n 1)" -o ci/binaries/matterbridge-$VERSION-darwin-amd64
+cd ci
+cat > deploy.json <<EOF
+{
+    "package": {
+        "name": "Matterbridge",
+        "repo": "nightly",
+        "subject": "42wim"
+    },
+    "version": {
+        "name": "$VERSION"
+    },
+    "files":
+        [
+        {"includePattern": "ci/binaries/(.*)", "uploadPattern":"\$1"}
+        ],
+    "publish": true
+}
+EOF
+

gateway/gateway.go

@@ -6,6 +6,8 @@ import (
 	"github.com/42wim/matterbridge/bridge/config"
 	log "github.com/Sirupsen/logrus"
 	//	"github.com/davecgh/go-spew/spew"
+	"github.com/hashicorp/golang-lru"
+	"github.com/peterhellberg/emojilib"
 	"regexp"
 	"strings"
 	"time"
@@ -13,62 +15,41 @@ import (
 
 type Gateway struct {
 	*config.Config
-	MyConfig        *config.Gateway
-	Bridges         map[string]*bridge.Bridge
-	Channels        map[string]*config.ChannelInfo
-	ChannelOptions  map[string]config.ChannelOptions
-	Names           map[string]bool
-	Name            string
-	Message         chan config.Message
-	DestChannelFunc func(msg *config.Message, dest bridge.Bridge) []config.ChannelInfo
+	Router         *Router
+	MyConfig       *config.Gateway
+	Bridges        map[string]*bridge.Bridge
+	Channels       map[string]*config.ChannelInfo
+	ChannelOptions map[string]config.ChannelOptions
+	Message        chan config.Message
+	Name           string
+	Messages       *lru.Cache
 }
 
-func New(cfg *config.Config) *Gateway {
-	gw := &Gateway{}
-	gw.Config = cfg
-	gw.Channels = make(map[string]*config.ChannelInfo)
-	gw.Message = make(chan config.Message)
-	gw.Bridges = make(map[string]*bridge.Bridge)
-	gw.Names = make(map[string]bool)
-	gw.DestChannelFunc = gw.getDestChannel
+type BrMsgID struct {
+	br *bridge.Bridge
+	ID string
+}
+
+func New(cfg config.Gateway, r *Router) *Gateway {
+	gw := &Gateway{Channels: make(map[string]*config.ChannelInfo), Message: r.Message,
+		Router: r, Bridges: make(map[string]*bridge.Bridge), Config: r.Config}
+	cache, _ := lru.New(5000)
+	gw.Messages = cache
+	gw.AddConfig(&cfg)
 	return gw
 }
 
 func (gw *Gateway) AddBridge(cfg *config.Bridge) error {
-	for _, br := range gw.Bridges {
-		if br.Account == cfg.Account {
-			gw.mapChannelsToBridge(br)
-			err := br.JoinChannels()
-			if err != nil {
-				return fmt.Errorf("Bridge %s failed to join channel: %v", br.Account, err)
-			}
-			return nil
-		}
+	br := gw.Router.getBridge(cfg.Account)
+	if br == nil {
+		br = bridge.New(gw.Config, cfg, gw.Message)
 	}
-	log.Infof("Starting bridge: %s ", cfg.Account)
-	br := bridge.New(gw.Config, cfg, gw.Message)
 	gw.mapChannelsToBridge(br)
 	gw.Bridges[cfg.Account] = br
-	err := br.Connect()
-	if err != nil {
-		return fmt.Errorf("Bridge %s failed to start: %v", br.Account, err)
-	}
-	err = br.JoinChannels()
-	if err != nil {
-		return fmt.Errorf("Bridge %s failed to join channel: %v", br.Account, err)
-	}
 	return nil
 }
 
 func (gw *Gateway) AddConfig(cfg *config.Gateway) error {
-	if gw.Names[cfg.Name] {
-		return fmt.Errorf("Gateway with name %s already exists", cfg.Name)
-	}
-	if cfg.Name == "" {
-		return fmt.Errorf("%s", "Gateway without name found")
-	}
-	log.Infof("Starting gateway: %s", cfg.Name)
-	gw.Names[cfg.Name] = true
 	gw.Name = cfg.Name
 	gw.MyConfig = cfg
 	gw.mapChannels()
@@ -89,41 +70,6 @@ func (gw *Gateway) mapChannelsToBridge(br *bridge.Bridge) {
 	}
 }
 
-func (gw *Gateway) Start() error {
-	go gw.handleReceive()
-	return nil
-}
-
-func (gw *Gateway) handleReceive() {
-	for {
-		select {
-		case msg := <-gw.Message:
-			if msg.Event == config.EVENT_FAILURE {
-				for _, br := range gw.Bridges {
-					if msg.Account == br.Account {
-						go gw.reconnectBridge(br)
-					}
-				}
-			}
-			if msg.Event == config.EVENT_REJOIN_CHANNELS {
-				for _, br := range gw.Bridges {
-					if msg.Account == br.Account {
-						br.Joined = make(map[string]bool)
-						br.JoinChannels()
-					}
-				}
-				continue
-			}
-			if !gw.ignoreMessage(&msg) {
-				msg.Timestamp = time.Now()
-				for _, br := range gw.Bridges {
-					gw.handleMessage(msg, br)
-				}
-			}
-		}
-	}
-}
-
 func (gw *Gateway) reconnectBridge(br *bridge.Bridge) {
 	br.Disconnect()
 	time.Sleep(time.Second * 5)
@@ -139,51 +85,51 @@ RECONNECT:
 	br.JoinChannels()
 }
 
-func (gw *Gateway) mapChannels() error {
-	for _, br := range append(gw.MyConfig.Out, gw.MyConfig.InOut...) {
+func (gw *Gateway) mapChannelConfig(cfg []config.Bridge, direction string) {
+	for _, br := range cfg {
 		if isApi(br.Account) {
 			br.Channel = "api"
 		}
 		ID := br.Channel + br.Account
-		_, ok := gw.Channels[ID]
-		if !ok {
-			channel := &config.ChannelInfo{Name: br.Channel, Direction: "out", ID: ID, Options: br.Options, Account: br.Account,
-				GID: make(map[string]bool), SameChannel: make(map[string]bool)}
-			channel.GID[gw.Name] = true
+		if _, ok := gw.Channels[ID]; !ok {
+			channel := &config.ChannelInfo{Name: br.Channel, Direction: direction, ID: ID, Options: br.Options, Account: br.Account,
+				SameChannel: make(map[string]bool)}
 			channel.SameChannel[gw.Name] = br.SameChannel
 			gw.Channels[channel.ID] = channel
+		} else {
+			// if we already have a key and it's not our current direction it means we have a bidirectional inout
+			if gw.Channels[ID].Direction != direction {
+				gw.Channels[ID].Direction = "inout"
+			}
 		}
-		gw.Channels[ID].GID[gw.Name] = true
 		gw.Channels[ID].SameChannel[gw.Name] = br.SameChannel
 	}
+}
 
-	for _, br := range append(gw.MyConfig.In, gw.MyConfig.InOut...) {
-		if isApi(br.Account) {
-			br.Channel = "api"
-		}
-		ID := br.Channel + br.Account
-		_, ok := gw.Channels[ID]
-		if !ok {
-			channel := &config.ChannelInfo{Name: br.Channel, Direction: "in", ID: ID, Options: br.Options, Account: br.Account,
-				GID: make(map[string]bool), SameChannel: make(map[string]bool)}
-			channel.GID[gw.Name] = true
-			channel.SameChannel[gw.Name] = br.SameChannel
-			gw.Channels[channel.ID] = channel
-		}
-		gw.Channels[ID].GID[gw.Name] = true
-		gw.Channels[ID].SameChannel[gw.Name] = br.SameChannel
-	}
+func (gw *Gateway) mapChannels() error {
+	gw.mapChannelConfig(gw.MyConfig.In, "in")
+	gw.mapChannelConfig(gw.MyConfig.Out, "out")
+	gw.mapChannelConfig(gw.MyConfig.InOut, "inout")
 	return nil
 }
 
 func (gw *Gateway) getDestChannel(msg *config.Message, dest bridge.Bridge) []config.ChannelInfo {
 	var channels []config.ChannelInfo
+	// if source channel is in only, do nothing
+	for _, channel := range gw.Channels {
+		// lookup the channel from the message
+		if channel.ID == getChannelID(*msg) {
+			// we only have destinations if the original message is from an "in" (sending) channel
+			if !strings.Contains(channel.Direction, "in") {
+				return channels
+			}
+			continue
+		}
+	}
 	for _, channel := range gw.Channels {
 		if _, ok := gw.Channels[getChannelID(*msg)]; !ok {
 			continue
 		}
-		// add gateway to message
-		gw.validGatewayDest(msg, channel)
 
 		// do samechannelgateway logic
 		if channel.SameChannel[msg.Gateway] {
@@ -192,48 +138,82 @@ func (gw *Gateway) getDestChannel(msg *config.Message, dest bridge.Bridge) []con
 			}
 			continue
 		}
-
-		if channel.Direction == "out" && channel.Account == dest.Account && gw.validGatewayDest(msg, channel) {
+		if strings.Contains(channel.Direction, "out") && channel.Account == dest.Account && gw.validGatewayDest(msg, channel) {
 			channels = append(channels, *channel)
 		}
 	}
 	return channels
 }
 
-func (gw *Gateway) handleMessage(msg config.Message, dest *bridge.Bridge) {
+func (gw *Gateway) handleMessage(msg config.Message, dest *bridge.Bridge) []*BrMsgID {
+	var brMsgIDs []*BrMsgID
+
+	// TODO refactor
+	// only slack now, check will have to be done in the different bridges.
+	// we need to check if we can't use fallback or text in other bridges
+	if msg.Extra != nil {
+		if dest.Protocol != "slack" {
+			if msg.Text == "" {
+				return brMsgIDs
+			}
+		}
+	}
 	// only relay join/part when configged
 	if msg.Event == config.EVENT_JOIN_LEAVE && !gw.Bridges[dest.Account].Config.ShowJoinPart {
-		return
+		return brMsgIDs
 	}
 	// broadcast to every out channel (irc QUIT)
 	if msg.Channel == "" && msg.Event != config.EVENT_JOIN_LEAVE {
 		log.Debug("empty channel")
-		return
+		return brMsgIDs
 	}
 	originchannel := msg.Channel
 	origmsg := msg
-	for _, channel := range gw.DestChannelFunc(&msg, *dest) {
+	channels := gw.getDestChannel(&msg, *dest)
+	for _, channel := range channels {
 		// do not send to ourself
 		if channel.ID == getChannelID(origmsg) {
 			continue
 		}
 		log.Debugf("Sending %#v from %s (%s) to %s (%s)", msg, msg.Account, originchannel, dest.Account, channel.Name)
 		msg.Channel = channel.Name
-		gw.modifyAvatar(&msg, dest)
-		gw.modifyUsername(&msg, dest)
+		msg.Avatar = gw.modifyAvatar(origmsg, dest)
+		msg.Username = gw.modifyUsername(origmsg, dest)
+		msg.ID = ""
+		if res, ok := gw.Messages.Get(origmsg.ID); ok {
+			IDs := res.([]*BrMsgID)
+			for _, id := range IDs {
+				if dest.Protocol == id.br.Protocol {
+					msg.ID = id.ID
+				}
+			}
+		}
 		// for api we need originchannel as channel
 		if dest.Protocol == "api" {
 			msg.Channel = originchannel
 		}
-		err := dest.Send(msg)
+		mID, err := dest.Send(msg)
 		if err != nil {
 			fmt.Println(err)
 		}
+		// append the message ID (mID) from this bridge (dest) to our brMsgIDs slice
+		if mID != "" {
+			brMsgIDs = append(brMsgIDs, &BrMsgID{dest, mID})
+		}
 	}
+	return brMsgIDs
 }
 
 func (gw *Gateway) ignoreMessage(msg *config.Message) bool {
+	// if we don't have the bridge, ignore it
+	if _, ok := gw.Bridges[msg.Account]; !ok {
+		return true
+	}
 	if msg.Text == "" {
+		// we have an attachment
+		if msg.Extra != nil && msg.Extra["attachments"] != nil {
+			return false
+		}
 		log.Debugf("ignoring empty message %#v from %s", msg, msg.Account)
 		return true
 	}
@@ -260,7 +240,7 @@ func (gw *Gateway) ignoreMessage(msg *config.Message) bool {
 	return false
 }
 
-func (gw *Gateway) modifyUsername(msg *config.Message, dest *bridge.Bridge) {
+func (gw *Gateway) modifyUsername(msg config.Message, dest *bridge.Bridge) string {
 	br := gw.Bridges[msg.Account]
 	msg.Protocol = br.Protocol
 	nick := gw.Config.General.RemoteNickFormat
@@ -282,10 +262,10 @@ func (gw *Gateway) modifyUsername(msg *config.Message, dest *bridge.Bridge) {
 	nick = strings.Replace(nick, "{NICK}", msg.Username, -1)
 	nick = strings.Replace(nick, "{BRIDGE}", br.Name, -1)
 	nick = strings.Replace(nick, "{PROTOCOL}", br.Protocol, -1)
-	msg.Username = nick
+	return nick
 }
 
-func (gw *Gateway) modifyAvatar(msg *config.Message, dest *bridge.Bridge) {
+func (gw *Gateway) modifyAvatar(msg config.Message, dest *bridge.Bridge) string {
 	iconurl := gw.Config.General.IconURL
 	if iconurl == "" {
 		iconurl = dest.Config.IconURL
@@ -294,6 +274,13 @@ func (gw *Gateway) modifyAvatar(msg *config.Message, dest *bridge.Bridge) {
 	if msg.Avatar == "" {
 		msg.Avatar = iconurl
 	}
+	return msg.Avatar
+}
+
+func (gw *Gateway) modifyMessage(msg *config.Message) {
+	// replace :emoji: to unicode
+	msg.Text = emojilib.Replace(msg.Text)
+	msg.Gateway = gw.Name
 }
 
 func getChannelID(msg config.Message) string {
@@ -301,40 +288,9 @@ func getChannelID(msg config.Message) string {
 }
 
 func (gw *Gateway) validGatewayDest(msg *config.Message, channel *config.ChannelInfo) bool {
-	GIDmap := gw.Channels[getChannelID(*msg)].GID
-
-	// gateway is specified in message (probably from api)
-	if msg.Gateway != "" {
-		return channel.GID[msg.Gateway]
-	}
-
-	// check if we are running a samechannelgateway.
-	// if it is and the channel name matches it's ok, otherwise we shouldn't use this channel.
-	for k, _ := range GIDmap {
-		if channel.SameChannel[k] == true {
-			if msg.Channel == channel.Name {
-				// add the gateway to our message
-				msg.Gateway = k
-				return true
-			} else {
-				return false
-			}
-		}
-	}
-	// check if we are in the correct gateway
-	for k, _ := range GIDmap {
-		if channel.GID[k] == true {
-			// add the gateway to our message
-			msg.Gateway = k
-			return true
-		}
-	}
-	return false
+	return msg.Gateway == gw.Name
 }
 
 func isApi(account string) bool {
-	if strings.HasPrefix(account, "api.") {
-		return true
-	}
-	return false
+	return strings.HasPrefix(account, "api.")
 }

gateway/gateway_test.go

@@ -0,0 +1,288 @@
+package gateway
+
+import (
+	"fmt"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/BurntSushi/toml"
+	"github.com/stretchr/testify/assert"
+	"strconv"
+
+	"testing"
+)
+
+var testconfig = `
+[irc.freenode]
+[mattermost.test]
+[gitter.42wim]
+[discord.test]
+[slack.test]
+
+[[gateway]]
+    name = "bridge1"
+    enable=true
+    
+    [[gateway.inout]]
+    account = "irc.freenode"
+    channel = "#wimtesting"
+    
+    [[gateway.inout]]
+    account="gitter.42wim"
+    channel="42wim/testroom"
+    #channel="matterbridge/Lobby"
+
+    [[gateway.inout]]
+    account = "discord.test"
+    channel = "general"
+    
+    [[gateway.inout]]
+    account="slack.test"
+    channel="testing"
+	`
+
+var testconfig2 = `
+[irc.freenode]
+[mattermost.test]
+[gitter.42wim]
+[discord.test]
+[slack.test]
+
+[[gateway]]
+    name = "bridge1"
+    enable=true
+    
+    [[gateway.in]]
+    account = "irc.freenode"
+    channel = "#wimtesting"
+    
+    [[gateway.in]]
+    account="gitter.42wim"
+    channel="42wim/testroom"
+
+    [[gateway.inout]]
+    account = "discord.test"
+    channel = "general"
+    
+    [[gateway.out]]
+    account="slack.test"
+    channel="testing"
+[[gateway]]
+    name = "bridge2"
+    enable=true
+    
+    [[gateway.in]]
+    account = "irc.freenode"
+    channel = "#wimtesting2"
+    
+    [[gateway.out]]
+    account="gitter.42wim"
+    channel="42wim/testroom"
+
+    [[gateway.out]]
+    account = "discord.test"
+    channel = "general2"
+	`
+var testconfig3 = `
+[irc.zzz]
+[telegram.zzz]
+[slack.zzz]
+[[gateway]]
+name="bridge"
+enable=true
+
+    [[gateway.inout]]
+    account="irc.zzz"
+    channel="#main"		
+
+    [[gateway.inout]]
+    account="telegram.zzz"
+    channel="-1111111111111"
+
+    [[gateway.inout]]
+    account="slack.zzz"
+    channel="irc"	
+	
+[[gateway]]
+name="announcements"
+enable=true
+	
+    [[gateway.in]]
+    account="telegram.zzz"
+    channel="-2222222222222"	
+	
+    [[gateway.out]]
+    account="irc.zzz"
+    channel="#main"		
+	
+    [[gateway.out]]
+    account="irc.zzz"
+    channel="#main-help"	
+
+    [[gateway.out]]
+    account="telegram.zzz"
+    channel="--333333333333"	
+
+    [[gateway.out]]
+    account="slack.zzz"
+    channel="general"		
+	
+[[gateway]]
+name="bridge2"
+enable=true
+
+    [[gateway.inout]]
+    account="irc.zzz"
+    channel="#main-help"	
+
+    [[gateway.inout]]
+    account="telegram.zzz"
+    channel="--444444444444"	
+
+	
+[[gateway]]
+name="bridge3"
+enable=true
+
+    [[gateway.inout]]
+    account="irc.zzz"
+    channel="#main-telegram"	
+
+    [[gateway.inout]]
+    account="telegram.zzz"
+    channel="--333333333333"
+`
+
+func maketestRouter(input string) *Router {
+	var cfg *config.Config
+	if _, err := toml.Decode(input, &cfg); err != nil {
+		fmt.Println(err)
+	}
+	r, err := NewRouter(cfg)
+	if err != nil {
+		fmt.Println(err)
+	}
+	return r
+}
+func TestNewRouter(t *testing.T) {
+	var cfg *config.Config
+	if _, err := toml.Decode(testconfig, &cfg); err != nil {
+		fmt.Println(err)
+	}
+	r, err := NewRouter(cfg)
+	if err != nil {
+		fmt.Println(err)
+	}
+	assert.Equal(t, 1, len(r.Gateways))
+	assert.Equal(t, 4, len(r.Gateways["bridge1"].Bridges))
+	assert.Equal(t, 4, len(r.Gateways["bridge1"].Channels))
+
+	r = maketestRouter(testconfig2)
+	assert.Equal(t, 2, len(r.Gateways))
+	assert.Equal(t, 4, len(r.Gateways["bridge1"].Bridges))
+	assert.Equal(t, 3, len(r.Gateways["bridge2"].Bridges))
+	assert.Equal(t, 4, len(r.Gateways["bridge1"].Channels))
+	assert.Equal(t, 3, len(r.Gateways["bridge2"].Channels))
+	assert.Equal(t, &config.ChannelInfo{Name: "42wim/testroom", Direction: "out",
+		ID: "42wim/testroomgitter.42wim", Account: "gitter.42wim",
+		SameChannel: map[string]bool{"bridge2": false}},
+		r.Gateways["bridge2"].Channels["42wim/testroomgitter.42wim"])
+	assert.Equal(t, &config.ChannelInfo{Name: "42wim/testroom", Direction: "in",
+		ID: "42wim/testroomgitter.42wim", Account: "gitter.42wim",
+		SameChannel: map[string]bool{"bridge1": false}},
+		r.Gateways["bridge1"].Channels["42wim/testroomgitter.42wim"])
+	assert.Equal(t, &config.ChannelInfo{Name: "general", Direction: "inout",
+		ID: "generaldiscord.test", Account: "discord.test",
+		SameChannel: map[string]bool{"bridge1": false}},
+		r.Gateways["bridge1"].Channels["generaldiscord.test"])
+}
+
+func TestGetDestChannel(t *testing.T) {
+	r := maketestRouter(testconfig2)
+	msg := &config.Message{Text: "test", Channel: "general", Account: "discord.test", Gateway: "bridge1", Protocol: "discord", Username: "test"}
+	for _, br := range r.Gateways["bridge1"].Bridges {
+		switch br.Account {
+		case "discord.test":
+			assert.Equal(t, []config.ChannelInfo{{Name: "general", Account: "discord.test", Direction: "inout", ID: "generaldiscord.test", SameChannel: map[string]bool{"bridge1": false}, Options: config.ChannelOptions{Key: ""}}},
+				r.Gateways["bridge1"].getDestChannel(msg, *br))
+		case "slack.test":
+			assert.Equal(t, []config.ChannelInfo{{Name: "testing", Account: "slack.test", Direction: "out", ID: "testingslack.test", SameChannel: map[string]bool{"bridge1": false}, Options: config.ChannelOptions{Key: ""}}},
+				r.Gateways["bridge1"].getDestChannel(msg, *br))
+		case "gitter.42wim":
+			assert.Equal(t, []config.ChannelInfo(nil), r.Gateways["bridge1"].getDestChannel(msg, *br))
+		case "irc.freenode":
+			assert.Equal(t, []config.ChannelInfo(nil), r.Gateways["bridge1"].getDestChannel(msg, *br))
+		}
+	}
+}
+
+func TestGetDestChannelAdvanced(t *testing.T) {
+	r := maketestRouter(testconfig3)
+	var msgs []*config.Message
+	i := 0
+	for _, gw := range r.Gateways {
+		for _, channel := range gw.Channels {
+			msgs = append(msgs, &config.Message{Text: "text" + strconv.Itoa(i), Channel: channel.Name, Account: channel.Account, Gateway: gw.Name, Username: "user" + strconv.Itoa(i)})
+			i++
+		}
+	}
+	hits := make(map[string]int)
+	for _, gw := range r.Gateways {
+		for _, br := range gw.Bridges {
+			for _, msg := range msgs {
+				channels := gw.getDestChannel(msg, *br)
+				if gw.Name != msg.Gateway {
+					assert.Equal(t, []config.ChannelInfo(nil), channels)
+					continue
+				}
+				switch gw.Name {
+				case "bridge":
+					if (msg.Channel == "#main" || msg.Channel == "-1111111111111" || msg.Channel == "irc") && (msg.Account == "irc.zzz" || msg.Account == "telegram.zzz" || msg.Account == "slack.zzz") {
+						hits[gw.Name]++
+						switch br.Account {
+						case "irc.zzz":
+							assert.Equal(t, []config.ChannelInfo{{Name: "#main", Account: "irc.zzz", Direction: "inout", ID: "#mainirc.zzz", SameChannel: map[string]bool{"bridge": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+						case "telegram.zzz":
+							assert.Equal(t, []config.ChannelInfo{{Name: "-1111111111111", Account: "telegram.zzz", Direction: "inout", ID: "-1111111111111telegram.zzz", SameChannel: map[string]bool{"bridge": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+						case "slack.zzz":
+							assert.Equal(t, []config.ChannelInfo{{Name: "irc", Account: "slack.zzz", Direction: "inout", ID: "ircslack.zzz", SameChannel: map[string]bool{"bridge": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+						}
+					}
+				case "bridge2":
+					if (msg.Channel == "#main-help" || msg.Channel == "--444444444444") && (msg.Account == "irc.zzz" || msg.Account == "telegram.zzz") {
+						hits[gw.Name]++
+						switch br.Account {
+						case "irc.zzz":
+							assert.Equal(t, []config.ChannelInfo{{Name: "#main-help", Account: "irc.zzz", Direction: "inout", ID: "#main-helpirc.zzz", SameChannel: map[string]bool{"bridge2": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+						case "telegram.zzz":
+							assert.Equal(t, []config.ChannelInfo{{Name: "--444444444444", Account: "telegram.zzz", Direction: "inout", ID: "--444444444444telegram.zzz", SameChannel: map[string]bool{"bridge2": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+						}
+					}
+				case "bridge3":
+					if (msg.Channel == "#main-telegram" || msg.Channel == "--333333333333") && (msg.Account == "irc.zzz" || msg.Account == "telegram.zzz") {
+						hits[gw.Name]++
+						switch br.Account {
+						case "irc.zzz":
+							assert.Equal(t, []config.ChannelInfo{{Name: "#main-telegram", Account: "irc.zzz", Direction: "inout", ID: "#main-telegramirc.zzz", SameChannel: map[string]bool{"bridge3": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+						case "telegram.zzz":
+							assert.Equal(t, []config.ChannelInfo{{Name: "--333333333333", Account: "telegram.zzz", Direction: "inout", ID: "--333333333333telegram.zzz", SameChannel: map[string]bool{"bridge3": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+						}
+					}
+				case "announcements":
+					if msg.Channel != "-2222222222222" && msg.Account != "telegram" {
+						assert.Equal(t, []config.ChannelInfo(nil), channels)
+						continue
+					}
+					hits[gw.Name]++
+					switch br.Account {
+					case "irc.zzz":
+						assert.Equal(t, []config.ChannelInfo{{Name: "#main", Account: "irc.zzz", Direction: "out", ID: "#mainirc.zzz", SameChannel: map[string]bool{"announcements": false}, Options: config.ChannelOptions{Key: ""}}, {Name: "#main-help", Account: "irc.zzz", Direction: "out", ID: "#main-helpirc.zzz", SameChannel: map[string]bool{"announcements": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+					case "slack.zzz":
+						assert.Equal(t, []config.ChannelInfo{{Name: "general", Account: "slack.zzz", Direction: "out", ID: "generalslack.zzz", SameChannel: map[string]bool{"announcements": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+					case "telegram.zzz":
+						assert.Equal(t, []config.ChannelInfo{{Name: "--333333333333", Account: "telegram.zzz", Direction: "out", ID: "--333333333333telegram.zzz", SameChannel: map[string]bool{"announcements": false}, Options: config.ChannelOptions{Key: ""}}}, channels)
+					}
+				}
+			}
+		}
+	}
+	assert.Equal(t, map[string]int{"bridge3": 4, "bridge": 9, "announcements": 3, "bridge2": 4}, hits)
+}

gateway/router.go

@@ -0,0 +1,112 @@
+package gateway
+
+import (
+	"fmt"
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/gateway/samechannel"
+	log "github.com/Sirupsen/logrus"
+	//	"github.com/davecgh/go-spew/spew"
+	"time"
+)
+
+type Router struct {
+	Gateways map[string]*Gateway
+	Message  chan config.Message
+	*config.Config
+}
+
+func NewRouter(cfg *config.Config) (*Router, error) {
+	r := &Router{}
+	r.Config = cfg
+	r.Message = make(chan config.Message)
+	r.Gateways = make(map[string]*Gateway)
+	sgw := samechannelgateway.New(cfg)
+	gwconfigs := sgw.GetConfig()
+
+	for _, entry := range append(gwconfigs, cfg.Gateway...) {
+		if !entry.Enable {
+			continue
+		}
+		if entry.Name == "" {
+			return nil, fmt.Errorf("%s", "Gateway without name found")
+		}
+		if _, ok := r.Gateways[entry.Name]; ok {
+			return nil, fmt.Errorf("Gateway with name %s already exists", entry.Name)
+		}
+		r.Gateways[entry.Name] = New(entry, r)
+	}
+	return r, nil
+}
+
+func (r *Router) Start() error {
+	m := make(map[string]*bridge.Bridge)
+	for _, gw := range r.Gateways {
+		for _, br := range gw.Bridges {
+			m[br.Account] = br
+		}
+	}
+	for _, br := range m {
+		log.Infof("Starting bridge: %s ", br.Account)
+		err := br.Connect()
+		if err != nil {
+			return fmt.Errorf("Bridge %s failed to start: %v", br.Account, err)
+		}
+		err = br.JoinChannels()
+		if err != nil {
+			return fmt.Errorf("Bridge %s failed to join channel: %v", br.Account, err)
+		}
+	}
+	go r.handleReceive()
+	return nil
+}
+
+func (r *Router) getBridge(account string) *bridge.Bridge {
+	for _, gw := range r.Gateways {
+		if br, ok := gw.Bridges[account]; ok {
+			return br
+		}
+	}
+	return nil
+}
+
+func (r *Router) handleReceive() {
+	for msg := range r.Message {
+		if msg.Event == config.EVENT_FAILURE {
+		Loop:
+			for _, gw := range r.Gateways {
+				for _, br := range gw.Bridges {
+					if msg.Account == br.Account {
+						go gw.reconnectBridge(br)
+						break Loop
+					}
+				}
+			}
+		}
+		if msg.Event == config.EVENT_REJOIN_CHANNELS {
+			for _, gw := range r.Gateways {
+				for _, br := range gw.Bridges {
+					if msg.Account == br.Account {
+						br.Joined = make(map[string]bool)
+						br.JoinChannels()
+					}
+				}
+			}
+		}
+		for _, gw := range r.Gateways {
+			// record all the message ID's of the different bridges
+			var msgIDs []*BrMsgID
+			if !gw.ignoreMessage(&msg) {
+				msg.Timestamp = time.Now()
+				gw.modifyMessage(&msg)
+				for _, br := range gw.Bridges {
+					msgIDs = append(msgIDs, gw.handleMessage(msg, br)...)
+				}
+				// only add the message ID if it doesn't already exists
+				if _, ok := gw.Messages.Get(msg.ID); !ok && msg.ID != "" {
+					gw.Messages.Add(msg.ID, msgIDs)
+				}
+			}
+		}
+	}
+}

gateway/samechannel/samechannel_test.go

@@ -0,0 +1,31 @@
+package samechannelgateway
+
+import (
+	"fmt"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/BurntSushi/toml"
+	"github.com/stretchr/testify/assert"
+
+	"testing"
+)
+
+var testconfig = `
+[mattermost.test]
+[slack.test]
+
+[[samechannelgateway]]
+   enable = true
+   name = "blah"
+      accounts = [ "mattermost.test","slack.test" ]
+      channels = [ "testing","testing2","testing10"]
+`
+
+func TestGetConfig(t *testing.T) {
+	var cfg *config.Config
+	if _, err := toml.Decode(testconfig, &cfg); err != nil {
+		fmt.Println(err)
+	}
+	sgw := New(cfg)
+	configs := sgw.GetConfig()
+	assert.Equal(t, []config.Gateway{{Name: "blah", Enable: true, In: []config.Bridge(nil), Out: []config.Bridge(nil), InOut: []config.Bridge{{Account: "mattermost.test", Channel: "testing", Options: config.ChannelOptions{Key: ""}, SameChannel: true}, {Account: "mattermost.test", Channel: "testing2", Options: config.ChannelOptions{Key: ""}, SameChannel: true}, {Account: "mattermost.test", Channel: "testing10", Options: config.ChannelOptions{Key: ""}, SameChannel: true}, {Account: "slack.test", Channel: "testing", Options: config.ChannelOptions{Key: ""}, SameChannel: true}, {Account: "slack.test", Channel: "testing2", Options: config.ChannelOptions{Key: ""}, SameChannel: true}, {Account: "slack.test", Channel: "testing10", Options: config.ChannelOptions{Key: ""}, SameChannel: true}}}}, configs)
+}

hook/rockethook/rockethook.go

@@ -99,10 +99,9 @@ func (c *Client) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 
 // Receive returns an incoming message from mattermost outgoing webhooks URL.
 func (c *Client) Receive() Message {
-	for {
-		select {
-		case msg := <-c.In:
-			return msg
-		}
+	var msg Message
+	for msg = range c.In {
+		return msg
 	}
+	return msg
 }

matterbridge.go

@@ -5,14 +5,13 @@ import (
 	"fmt"
 	"github.com/42wim/matterbridge/bridge/config"
 	"github.com/42wim/matterbridge/gateway"
-	"github.com/42wim/matterbridge/gateway/samechannel"
 	log "github.com/Sirupsen/logrus"
 	"github.com/google/gops/agent"
 	"strings"
 )
 
 var (
-	version = "0.16.0-dev"
+	version = "1.3.1"
 	githash string
 )
 
@@ -43,20 +42,11 @@ func main() {
 		log.Println("WARNING: THIS IS A DEVELOPMENT VERSION. Things may break.")
 	}
 	cfg := config.NewConfig(*flagConfig)
-
-	g := gateway.New(cfg)
-	sgw := samechannelgateway.New(cfg)
-	gwconfigs := sgw.GetConfig()
-	for _, gw := range append(gwconfigs, cfg.Gateway...) {
-		if !gw.Enable {
-			continue
-		}
-		err := g.AddConfig(&gw)
-		if err != nil {
-			log.Fatalf("Starting gateway failed: %s", err)
-		}
+	r, err := gateway.NewRouter(cfg)
+	if err != nil {
+		log.Fatalf("Starting gateway failed: %s", err)
 	}
-	err := g.Start()
+	err = r.Start()
 	if err != nil {
 		log.Fatalf("Starting gateway failed: %s", err)
 	}

matterbridge.toml.sample

@@ -32,16 +32,38 @@ UseSASL=false
 #OPTIONAL (default false)
 SkipTLSVerify=true
 
+#If you know your charset, you can specify it manually. 
+#Otherwise it tries to detect this automatically. Select one below
+# "iso-8859-2:1987", "iso-8859-9:1989", "866", "latin9", "iso-8859-10:1992", "iso-ir-109", "hebrew", 
+# "cp932", "iso-8859-15", "cp437", "utf-16be", "iso-8859-3:1988", "windows-1251", "utf16", "latin6", 
+# "latin3", "iso-8859-1:1987", "iso-8859-9", "utf-16le", "big5", "cp819", "asmo-708", "utf-8", 
+# "ibm437", "iso-ir-157", "iso-ir-144", "latin4", "850", "iso-8859-5", "iso-8859-5:1988", "l3", 
+# "windows-31j", "utf8", "iso-8859-3", "437", "greek", "iso-8859-8", "l6", "l9-iso-8859-15", 
+# "iso-8859-2", "latin2", "iso-ir-100", "iso-8859-6", "arabic", "iso-ir-148", "us-ascii", "x-sjis", 
+# "utf16be", "iso-8859-8:1988", "utf16le", "l4", "utf-16", "iso-ir-138", "iso-8859-7", "iso-8859-7:1987", 
+# "windows-1252", "l2", "koi8-r", "iso8859-1", "latin1", "ecma-114", "iso-ir-110", "elot-928", 
+# "iso-ir-126", "iso-8859-1", "iso-ir-127", "cp850", "cyrillic", "greek8", "windows-1250", "iso-latin-1", 
+# "l5", "ibm866", "cp866", "ms-kanji", "ibm850", "ecma-118", "iso-ir-101", "ibm819", "l1", "iso-8859-6:1987", 
+# "latin5", "ascii", "sjis", "iso-8859-10", "iso-8859-4", "iso-8859-4:1988", "shift-jis
+# The select charset will be converted to utf-8 when sent to other bridges.
+#OPTIONAL (default "")
+Charset=""
+
 #Your nick on irc. 
 #REQUIRED
 Nick="matterbot"
 
 #If you registered your bot with a service like Nickserv on freenode. 
 #Also being used when UseSASL=true
+#
+#Note: if you want do to quakenet auth, set NickServNick="Q@CServe.quakenet.org"
 #OPTIONAL
 NickServNick="nickserv"
 NickServPassword="secret"
 
+#OPTIONAL only used for quakenet auth
+NickServUsername="username"
+
 #Flood control
 #Delay in milliseconds between each message send to the IRC server
 #OPTIONAL (default 1300)
@@ -213,6 +235,11 @@ Team="yourteam"
 Login="yourlogin"
 Password="yourpass"
 
+#personal access token of the bot.
+#new feature since mattermost 4.1. See https://docs.mattermost.com/developer/personal-access-tokens.html
+#OPTIONAL (you can use token instead of login/password)
+#Token="abcdefghijklm"
+
 #Enable this to make a http connection (instead of https) to your mattermost. 
 #OPTIONAL (default false)
 NoTLS=false
@@ -361,7 +388,6 @@ WebhookURL="https://hooks.slack.com/services/yourhook"
 #AND DEDICATED BOT USER WHEN POSSIBLE!
 #Address to listen on for outgoing webhook requests from slack
 #See account settings - integrations - outgoing webhooks on slack
-#This setting will not be used when useAPI is eanbled
 #webhooks
 #OPTIONAL
 WebhookBindAddress="0.0.0.0:9999"
@@ -443,7 +469,12 @@ Server="yourservername"
 #OPTIONAL (default false)
 ShowEmbeds=false
 
+#Shows the username (minus the discriminator) instead of the server nickname
+#OPTIONAL (default false)
+UseUserName=false
+
 #Specify WebhookURL. If given, will relay messages using the Webhook, which gives a better look to messages.
+#This only works if you have one discord channel, if you have multiple discord channels you'll have to specify it in the gateway config
 #OPTIONAL (default empty)
 WebhookURL="Yourwebhooktokenhere"
 
@@ -503,6 +534,12 @@ MessageFormat=""
 #OPTIONAL (default false)
 UseFirstName=false
 
+#WARNING! If enabled this will relay GIF/stickers/documents and other attachments as URLs
+#Those URLs will contain your bot-token. This may not be what you want.
+#For now there is no secure way to relay GIF/stickers/documents without seeing your token.
+#OPTIONAL (default false)
+UseInsecureURL=false
+
 #Disable sending of edits to other bridges
 #OPTIONAL (default false)
 EditDisable=false
@@ -823,6 +860,14 @@ enable=true
         #OPTIONAL - your irc channel key
         key="yourkey"
 
+    [[gateway.inout]]
+    account="discord.game"
+    channel="mygreatgame"
+
+        #OPTIONAL - webhookurl only works for discord (it needs a different URL for each cahnnel)
+        [gateway.inout.options]
+        webhookurl=""https://discordapp.com/api/webhooks/123456789123456789/C9WPqExYWONPDZabcdef-def1434FGFjstasJX9pYht73y"
+
     #API example
     #[[gateway.inout]]
     #account="api.local"

matterbridge.toml.simple

@@ -6,7 +6,6 @@
 
 [mattermost]
     [mattermost.work]
-    useAPI=true
     #do not prefix it wit http:// or https://
     Server="yourmattermostserver.domain" 
     Team="yourteam"

matterclient/matterclient.go

@@ -1,6 +1,7 @@
 package matterclient
 
 import (
+	"crypto/md5"
 	"crypto/tls"
 	"encoding/json"
 	"errors"
@@ -8,7 +9,6 @@ import (
 	"net/http"
 	"net/http/cookiejar"
 	"net/url"
-	"strconv"
 	"strings"
 	"sync"
 	"time"
@@ -16,6 +16,7 @@ import (
 	log "github.com/Sirupsen/logrus"
 
 	"github.com/gorilla/websocket"
+	"github.com/hashicorp/golang-lru"
 	"github.com/jpillora/backoff"
 	"github.com/mattermost/platform/model"
 )
@@ -43,8 +44,8 @@ type Message struct {
 type Team struct {
 	Team         *model.Team
 	Id           string
-	Channels     *model.ChannelList
-	MoreChannels *model.ChannelList
+	Channels     []*model.Channel
+	MoreChannels []*model.Channel
 	Users        map[string]*model.User
 }
 
@@ -53,7 +54,7 @@ type MMClient struct {
 	*Credentials
 	Team          *Team
 	OtherTeams    []*Team
-	Client        *model.Client
+	Client        *model.Client4
 	User          *model.User
 	Users         map[string]*model.User
 	MessageChan   chan *Message
@@ -65,6 +66,8 @@ type MMClient struct {
 	WsSequence    int64
 	WsPingChan    chan *model.WebSocketResponse
 	ServerVersion string
+	OnWsConnect   func()
+	lruCache      *lru.Cache
 }
 
 func New(login, pass, team, server string) *MMClient {
@@ -72,6 +75,7 @@ func New(login, pass, team, server string) *MMClient {
 	mmclient := &MMClient{Credentials: cred, MessageChan: make(chan *Message, 100), Users: make(map[string]*model.User)}
 	mmclient.log = log.WithFields(log.Fields{"module": "matterclient"})
 	log.SetFormatter(&log.TextFormatter{FullTimestamp: true})
+	mmclient.lruCache, _ = lru.New(500)
 	return mmclient
 }
 
@@ -87,7 +91,7 @@ func (m *MMClient) SetLogLevel(level string) {
 func (m *MMClient) Login() error {
 	// check if this is a first connect or a reconnection
 	firstConnection := true
-	if m.WsConnected == true {
+	if m.WsConnected {
 		firstConnection = false
 	}
 	m.WsConnected = false
@@ -100,24 +104,25 @@ func (m *MMClient) Login() error {
 		Jitter: true,
 	}
 	uriScheme := "https://"
-	wsScheme := "wss://"
 	if m.NoTLS {
 		uriScheme = "http://"
-		wsScheme = "ws://"
 	}
 	// login to mattermost
-	m.Client = model.NewClient(uriScheme + m.Credentials.Server)
+	m.Client = model.NewAPIv4Client(uriScheme + m.Credentials.Server)
 	m.Client.HttpClient.Transport = &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: m.SkipTLSVerify}, Proxy: http.ProxyFromEnvironment}
 	m.Client.HttpClient.Timeout = time.Second * 10
 
 	for {
 		d := b.Duration()
 		// bogus call to get the serverversion
-		m.Client.GetClientProperties()
-		if firstConnection && !supportedVersion(m.Client.ServerVersion) {
-			return fmt.Errorf("unsupported mattermost version: %s", m.Client.ServerVersion)
+		_, resp := m.Client.Logout()
+		if resp.Error != nil {
+			return fmt.Errorf("%#v", resp.Error.Error())
 		}
-		m.ServerVersion = m.Client.ServerVersion
+		if firstConnection && !supportedVersion(resp.ServerVersion) {
+			return fmt.Errorf("unsupported mattermost version: %s", resp.ServerVersion)
+		}
+		m.ServerVersion = resp.ServerVersion
 		if m.ServerVersion == "" {
 			m.log.Debugf("Server not up yet, reconnecting in %s", d)
 			time.Sleep(d)
@@ -128,30 +133,33 @@ func (m *MMClient) Login() error {
 	}
 	b.Reset()
 
-	var myinfo *model.Result
+	var resp *model.Response
+	//var myinfo *model.Result
 	var appErr *model.AppError
 	var logmsg = "trying login"
 	for {
 		m.log.Debugf("%s %s %s %s", logmsg, m.Credentials.Team, m.Credentials.Login, m.Credentials.Server)
 		if strings.Contains(m.Credentials.Pass, model.SESSION_COOKIE_TOKEN) {
-			m.log.Debugf(logmsg+" with %s", model.SESSION_COOKIE_TOKEN)
+			m.log.Debugf(logmsg + " with token")
 			token := strings.Split(m.Credentials.Pass, model.SESSION_COOKIE_TOKEN+"=")
 			if len(token) != 2 {
 				return errors.New("incorrect MMAUTHTOKEN. valid input is MMAUTHTOKEN=yourtoken")
 			}
 			m.Client.HttpClient.Jar = m.createCookieJar(token[1])
-			m.Client.MockSession(token[1])
-			myinfo, appErr = m.Client.GetMe("")
-			if appErr != nil {
-				return errors.New(appErr.DetailedError)
+			m.Client.AuthToken = token[1]
+			m.Client.AuthType = model.HEADER_BEARER
+			m.User, resp = m.Client.GetMe("")
+			if resp.Error != nil {
+				return resp.Error
 			}
-			if myinfo.Data.(*model.User) == nil {
+			if m.User == nil {
 				m.log.Errorf("LOGIN TOKEN: %s is invalid", m.Credentials.Pass)
 				return errors.New("invalid " + model.SESSION_COOKIE_TOKEN)
 			}
 		} else {
-			myinfo, appErr = m.Client.Login(m.Credentials.Login, m.Credentials.Pass)
+			m.User, resp = m.Client.Login(m.Credentials.Login, m.Credentials.Pass)
 		}
+		appErr = resp.Error
 		if appErr != nil {
 			d := b.Duration()
 			m.log.Debug(appErr.DetailedError)
@@ -179,17 +187,34 @@ func (m *MMClient) Login() error {
 	if m.Team == nil {
 		return errors.New("team not found")
 	}
-	// set our team id as default route
-	m.Client.SetTeamId(m.Team.Id)
+
+	m.wsConnect()
+
+	return nil
+}
+
+func (m *MMClient) wsConnect() {
+	b := &backoff.Backoff{
+		Min:    time.Second,
+		Max:    5 * time.Minute,
+		Jitter: true,
+	}
+
+	m.WsConnected = false
+	wsScheme := "wss://"
+	if m.NoTLS {
+		wsScheme = "ws://"
+	}
 
 	// setup websocket connection
-	wsurl := wsScheme + m.Credentials.Server + model.API_URL_SUFFIX_V3 + "/users/websocket"
+	wsurl := wsScheme + m.Credentials.Server + model.API_URL_SUFFIX_V4 + "/websocket"
 	header := http.Header{}
 	header.Set(model.HEADER_AUTH, "BEARER "+m.Client.AuthToken)
 
 	m.log.Debugf("WsClient: making connection: %s", wsurl)
 	for {
 		wsDialer := &websocket.Dialer{Proxy: http.ProxyFromEnvironment, TLSClientConfig: &tls.Config{InsecureSkipVerify: m.SkipTLSVerify}}
+		var err error
 		m.WsClient, _, err = wsDialer.Dial(wsurl, header)
 		if err != nil {
 			d := b.Duration()
@@ -199,15 +224,12 @@ func (m *MMClient) Login() error {
 		}
 		break
 	}
-	b.Reset()
 
 	m.log.Debug("WsClient: connected")
 	m.WsSequence = 1
 	m.WsPingChan = make(chan *model.WebSocketResponse)
 	// only start to parse WS messages when login is completely done
 	m.WsConnected = true
-
-	return nil
 }
 
 func (m *MMClient) Logout() error {
@@ -215,9 +237,13 @@ func (m *MMClient) Logout() error {
 	m.WsQuit = true
 	m.WsClient.Close()
 	m.WsClient.UnderlyingConn().Close()
-	_, err := m.Client.Logout()
-	if err != nil {
-		return err
+	if strings.Contains(m.Credentials.Pass, model.SESSION_COOKIE_TOKEN) {
+		m.log.Debug("Not invalidating session in logout, credential is a token")
+		return nil
+	}
+	_, resp := m.Client.Logout()
+	if resp.Error != nil {
+		return resp.Error
 	}
 	return nil
 }
@@ -240,21 +266,31 @@ func (m *MMClient) WsReceiver() {
 		if _, rawMsg, err = m.WsClient.ReadMessage(); err != nil {
 			m.log.Error("error:", err)
 			// reconnect
-			m.Login()
+			m.wsConnect()
 		}
 
 		var event model.WebSocketEvent
 		if err := json.Unmarshal(rawMsg, &event); err == nil && event.IsValid() {
-			m.log.Debugf("WsReceiver: %#v", event)
+			m.log.Debugf("WsReceiver event: %#v", event)
 			msg := &Message{Raw: &event, Team: m.Credentials.Team}
 			m.parseMessage(msg)
-			m.MessageChan <- msg
+			// check if we didn't empty the message
+			if msg.Text != "" {
+				m.MessageChan <- msg
+				continue
+			}
+			// if we have file attached but the message is empty, also send it
+			if msg.Post != nil {
+				if msg.Text != "" || len(msg.Post.FileIds) > 0 || msg.Post.Type == "slack_attachment" {
+					m.MessageChan <- msg
+				}
+			}
 			continue
 		}
 
 		var response model.WebSocketResponse
 		if err := json.Unmarshal(rawMsg, &response); err == nil && response.IsValid() {
-			m.log.Debugf("WsReceiver: %#v", response)
+			m.log.Debugf("WsReceiver response: %#v", response)
 			m.parseResponse(response)
 			continue
 		}
@@ -263,7 +299,7 @@ func (m *MMClient) WsReceiver() {
 
 func (m *MMClient) parseMessage(rmsg *Message) {
 	switch rmsg.Raw.Event {
-	case model.WEBSOCKET_EVENT_POSTED, model.WEBSOCKET_EVENT_POST_EDITED:
+	case model.WEBSOCKET_EVENT_POSTED, model.WEBSOCKET_EVENT_POST_EDITED, model.WEBSOCKET_EVENT_POST_DELETED:
 		m.parseActionPost(rmsg)
 		/*
 			case model.ACTION_USER_REMOVED:
@@ -284,10 +320,18 @@ func (m *MMClient) parseResponse(rmsg model.WebSocketResponse) {
 }
 
 func (m *MMClient) parseActionPost(rmsg *Message) {
+	// add post to cache, if it already exists don't relay this again.
+	// this should fix reposts
+	if ok, _ := m.lruCache.ContainsOrAdd(digestString(rmsg.Raw.Data["post"].(string)), true); ok {
+		m.log.Debugf("message %#v in cache, not processing again", rmsg.Raw.Data["post"].(string))
+		rmsg.Text = ""
+		return
+	}
 	data := model.PostFromJson(strings.NewReader(rmsg.Raw.Data["post"].(string)))
 	// we don't have the user, refresh the userlist
 	if m.GetUser(data.UserId) == nil {
-		m.UpdateUsers()
+		m.log.Infof("User %s is not known, ignoring message %s", data)
+		return
 	}
 	rmsg.Username = m.GetUserName(data.UserId)
 	rmsg.Channel = m.GetChannelName(data.ChannelId)
@@ -309,37 +353,37 @@ func (m *MMClient) parseActionPost(rmsg *Message) {
 	}
 	rmsg.Text = data.Message
 	rmsg.Post = data
-	return
 }
 
 func (m *MMClient) UpdateUsers() error {
-	mmusers, err := m.Client.GetProfiles(0, 50000, "")
-	if err != nil {
-		return errors.New(err.DetailedError)
+	mmusers, resp := m.Client.GetUsers(0, 50000, "")
+	if resp.Error != nil {
+		return errors.New(resp.Error.DetailedError)
 	}
 	m.Lock()
-	m.Users = mmusers.Data.(map[string]*model.User)
+	for _, user := range mmusers {
+		m.Users[user.Id] = user
+	}
 	m.Unlock()
 	return nil
 }
 
 func (m *MMClient) UpdateChannels() error {
-	mmchannels, err := m.Client.GetChannels("")
-	if err != nil {
-		return errors.New(err.DetailedError)
-	}
-	var mmchannels2 *model.Result
-	if m.mmVersion() >= 3.08 {
-		mmchannels2, err = m.Client.GetMoreChannelsPage(0, 5000)
-	} else {
-		mmchannels2, err = m.Client.GetMoreChannels("")
-	}
-	if err != nil {
-		return errors.New(err.DetailedError)
+	mmchannels, resp := m.Client.GetChannelsForTeamForUser(m.Team.Id, m.User.Id, "")
+	if resp.Error != nil {
+		return errors.New(resp.Error.DetailedError)
 	}
 	m.Lock()
-	m.Team.Channels = mmchannels.Data.(*model.ChannelList)
-	m.Team.MoreChannels = mmchannels2.Data.(*model.ChannelList)
+	m.Team.Channels = mmchannels
+	m.Unlock()
+
+	mmchannels, resp = m.Client.GetPublicChannelsForTeam(m.Team.Id, 0, 5000, "")
+	if resp.Error != nil {
+		return errors.New(resp.Error.DetailedError)
+	}
+
+	m.Lock()
+	m.Team.MoreChannels = mmchannels
 	m.Unlock()
 	return nil
 }
@@ -348,9 +392,21 @@ func (m *MMClient) GetChannelName(channelId string) string {
 	m.RLock()
 	defer m.RUnlock()
 	for _, t := range m.OtherTeams {
-		for _, channel := range append(*t.Channels, *t.MoreChannels...) {
-			if channel.Id == channelId {
-				return channel.Name
+		if t == nil {
+			continue
+		}
+		if t.Channels != nil {
+			for _, channel := range t.Channels {
+				if channel.Id == channelId {
+					return channel.Name
+				}
+			}
+		}
+		if t.MoreChannels != nil {
+			for _, channel := range t.MoreChannels {
+				if channel.Id == channelId {
+					return channel.Name
+				}
 			}
 		}
 	}
@@ -365,7 +421,7 @@ func (m *MMClient) GetChannelId(name string, teamId string) string {
 	}
 	for _, t := range m.OtherTeams {
 		if t.Id == teamId {
-			for _, channel := range append(*t.Channels, *t.MoreChannels...) {
+			for _, channel := range append(t.Channels, t.MoreChannels...) {
 				if channel.Name == name {
 					return channel.Id
 				}
@@ -379,7 +435,7 @@ func (m *MMClient) GetChannelTeamId(id string) string {
 	m.RLock()
 	defer m.RUnlock()
 	for _, t := range append(m.OtherTeams, m.Team) {
-		for _, channel := range append(*t.Channels, *t.MoreChannels...) {
+		for _, channel := range append(t.Channels, t.MoreChannels...) {
 			if channel.Id == id {
 				return channel.TeamId
 			}
@@ -392,7 +448,7 @@ func (m *MMClient) GetChannelHeader(channelId string) string {
 	m.RLock()
 	defer m.RUnlock()
 	for _, t := range m.OtherTeams {
-		for _, channel := range append(*t.Channels, *t.MoreChannels...) {
+		for _, channel := range append(t.Channels, t.MoreChannels...) {
 			if channel.Id == channelId {
 				return channel.Header
 			}
@@ -402,55 +458,85 @@ func (m *MMClient) GetChannelHeader(channelId string) string {
 	return ""
 }
 
-func (m *MMClient) PostMessage(channelId string, text string) {
+func (m *MMClient) PostMessage(channelId string, text string) (string, error) {
 	post := &model.Post{ChannelId: channelId, Message: text}
-	m.Client.CreatePost(post)
+	res, resp := m.Client.CreatePost(post)
+	if resp.Error != nil {
+		return "", resp.Error
+	}
+	return res.Id, nil
+}
+
+func (m *MMClient) PostMessageWithFiles(channelId string, text string, fileIds []string) (string, error) {
+	post := &model.Post{ChannelId: channelId, Message: text, FileIds: fileIds}
+	res, resp := m.Client.CreatePost(post)
+	if resp.Error != nil {
+		return "", resp.Error
+	}
+	return res.Id, nil
+}
+
+func (m *MMClient) EditMessage(postId string, text string) (string, error) {
+	post := &model.Post{Message: text}
+	res, resp := m.Client.UpdatePost(postId, post)
+	if resp.Error != nil {
+		return "", resp.Error
+	}
+	return res.Id, nil
+}
+
+func (m *MMClient) DeleteMessage(postId string) error {
+	_, resp := m.Client.DeletePost(postId)
+	if resp.Error != nil {
+		return resp.Error
+	}
+	return nil
 }
 
 func (m *MMClient) JoinChannel(channelId string) error {
 	m.RLock()
 	defer m.RUnlock()
-	for _, c := range *m.Team.Channels {
+	for _, c := range m.Team.Channels {
 		if c.Id == channelId {
 			m.log.Debug("Not joining ", channelId, " already joined.")
 			return nil
 		}
 	}
 	m.log.Debug("Joining ", channelId)
-	_, err := m.Client.JoinChannel(channelId)
-	if err != nil {
-		return errors.New("failed to join")
+	_, resp := m.Client.AddChannelMember(channelId, m.User.Id)
+	if resp.Error != nil {
+		return resp.Error
 	}
 	return nil
 }
 
 func (m *MMClient) GetPostsSince(channelId string, time int64) *model.PostList {
-	res, err := m.Client.GetPostsSince(channelId, time)
-	if err != nil {
+	res, resp := m.Client.GetPostsSince(channelId, time)
+	if resp.Error != nil {
 		return nil
 	}
-	return res.Data.(*model.PostList)
+	return res
 }
 
 func (m *MMClient) SearchPosts(query string) *model.PostList {
-	res, err := m.Client.SearchPosts(query, false)
-	if err != nil {
+	res, resp := m.Client.SearchPosts(m.Team.Id, query, false)
+	if resp.Error != nil {
 		return nil
 	}
-	return res.Data.(*model.PostList)
+	return res
 }
 
 func (m *MMClient) GetPosts(channelId string, limit int) *model.PostList {
-	res, err := m.Client.GetPosts(channelId, 0, limit, "")
-	if err != nil {
+	res, resp := m.Client.GetPostsForChannel(channelId, 0, limit, "")
+	if resp.Error != nil {
 		return nil
 	}
-	return res.Data.(*model.PostList)
+	return res
 }
 
 func (m *MMClient) GetPublicLink(filename string) string {
-	res, err := m.Client.GetPublicLink(filename)
-	if err != nil {
+	res, resp := m.Client.GetFileLink(filename)
+	if resp.Error != nil {
 		return ""
 	}
 	return res
@@ -459,8 +545,27 @@ func (m *MMClient) GetPublicLink(filename string) string {
 func (m *MMClient) GetPublicLinks(filenames []string) []string {
 	var output []string
 	for _, f := range filenames {
-		res, err := m.Client.GetPublicLink(f)
-		if err != nil {
+		res, resp := m.Client.GetFileLink(f)
+		if resp.Error != nil {
+			continue
+		}
+		output = append(output, res)
+	}
+	return output
+}
+
+func (m *MMClient) GetFileLinks(filenames []string) []string {
+	uriScheme := "https://"
+	if m.NoTLS {
+		uriScheme = "http://"
+	}
+
+	var output []string
+	for _, f := range filenames {
+		res, resp := m.Client.GetFileLink(f)
+		if resp.Error != nil {
+			// public links is probably disabled, create the link ourselves
+			output = append(output, uriScheme+m.Credentials.Server+model.API_URL_SUFFIX_V3+"/files/"+f+"/get")
 			continue
 		}
 		output = append(output, res)
@@ -469,44 +574,43 @@ func (m *MMClient) GetPublicLinks(filenames []string) []string {
 }
 
 func (m *MMClient) UpdateChannelHeader(channelId string, header string) {
-	data := make(map[string]string)
-	data["channel_id"] = channelId
-	data["channel_header"] = header
+	channel := &model.Channel{Id: channelId, Header: header}
 	m.log.Debugf("updating channelheader %#v, %#v", channelId, header)
-	_, err := m.Client.UpdateChannelHeader(data)
-	if err != nil {
-		log.Error(err)
+	_, resp := m.Client.UpdateChannel(channel)
+	if resp.Error != nil {
+		log.Error(resp.Error)
 	}
 }
 
 func (m *MMClient) UpdateLastViewed(channelId string) {
 	m.log.Debugf("posting lastview %#v", channelId)
-	if m.mmVersion() >= 3.08 {
-		view := model.ChannelView{ChannelId: channelId}
-		res, _ := m.Client.ViewChannel(view)
-		if res == false {
-			m.log.Errorf("ChannelView update for %s failed", channelId)
-		}
-		return
-	}
-	_, err := m.Client.UpdateLastViewedAt(channelId, true)
-	if err != nil {
-		m.log.Error(err)
+	view := &model.ChannelView{ChannelId: channelId}
+	res, _ := m.Client.ViewChannel(m.User.Id, view)
+	if !res {
+		m.log.Errorf("ChannelView update for %s failed", channelId)
 	}
 }
 
+func (m *MMClient) UpdateUserNick(nick string) error {
+	user := m.User
+	user.Nickname = nick
+	_, resp := m.Client.UpdateUser(user)
+	if resp.Error != nil {
+		return resp.Error
+	}
+	return nil
+}
+
 func (m *MMClient) UsernamesInChannel(channelId string) []string {
-	res, err := m.Client.GetMyChannelMembers()
-	if err != nil {
-		m.log.Errorf("UsernamesInChannel(%s) failed: %s", channelId, err)
+	res, resp := m.Client.GetChannelMembers(channelId, 0, 50000, "")
+	if resp.Error != nil {
+		m.log.Errorf("UsernamesInChannel(%s) failed: %s", channelId, resp.Error)
 		return []string{}
 	}
-	members := res.Data.(*model.ChannelMembers)
+	allusers := m.GetUsers()
 	result := []string{}
-	for _, channel := range *members {
-		if channel.ChannelId == channelId {
-			result = append(result, m.GetUser(channel.UserId).Username)
-		}
+	for _, member := range *res {
+		result = append(result, allusers[member.UserId].Nickname)
 	}
 	return result
 }
@@ -530,22 +634,15 @@ func (m *MMClient) createCookieJar(token string) *cookiejar.Jar {
 func (m *MMClient) SendDirectMessage(toUserId string, msg string) {
 	m.log.Debugf("SendDirectMessage to %s, msg %s", toUserId, msg)
 	// create DM channel (only happens on first message)
-	_, err := m.Client.CreateDirectChannel(toUserId)
-	if err != nil {
-		m.log.Debugf("SendDirectMessage to %#v failed: %s", toUserId, err)
+	_, resp := m.Client.CreateDirectChannel(m.User.Id, toUserId)
+	if resp.Error != nil {
+		m.log.Debugf("SendDirectMessage to %#v failed: %s", toUserId, resp.Error)
 		return
 	}
 	channelName := model.GetDMNameFromIds(toUserId, m.User.Id)
 
 	// update our channels
-	mmchannels, err := m.Client.GetChannels("")
-	if err != nil {
-		m.log.Debug("SendDirectMessage: Couldn't update channels")
-		return
-	}
-	m.Lock()
-	m.Team.Channels = mmchannels.Data.(*model.ChannelList)
-	m.Unlock()
+	m.UpdateChannels()
 
 	// build & send the message
 	msg = strings.Replace(msg, "\r", "", -1)
@@ -571,10 +668,10 @@ func (m *MMClient) GetChannels() []*model.Channel {
 	defer m.RUnlock()
 	var channels []*model.Channel
 	// our primary team channels first
-	channels = append(channels, *m.Team.Channels...)
+	channels = append(channels, m.Team.Channels...)
 	for _, t := range m.OtherTeams {
 		if t.Id != m.Team.Id {
-			channels = append(channels, *t.Channels...)
+			channels = append(channels, t.Channels...)
 		}
 	}
 	return channels
@@ -586,7 +683,7 @@ func (m *MMClient) GetMoreChannels() []*model.Channel {
 	defer m.RUnlock()
 	var channels []*model.Channel
 	for _, t := range m.OtherTeams {
-		channels = append(channels, *t.MoreChannels...)
+		channels = append(channels, t.MoreChannels...)
 	}
 	return channels
 }
@@ -597,8 +694,10 @@ func (m *MMClient) GetTeamFromChannel(channelId string) string {
 	defer m.RUnlock()
 	var channels []*model.Channel
 	for _, t := range m.OtherTeams {
-		channels = append(channels, *t.Channels...)
-		channels = append(channels, *t.MoreChannels...)
+		channels = append(channels, t.Channels...)
+		if t.MoreChannels != nil {
+			channels = append(channels, t.MoreChannels...)
+		}
 		for _, c := range channels {
 			if c.Id == channelId {
 				return t.Id
@@ -611,12 +710,11 @@ func (m *MMClient) GetTeamFromChannel(channelId string) string {
 func (m *MMClient) GetLastViewedAt(channelId string) int64 {
 	m.RLock()
 	defer m.RUnlock()
-	res, err := m.Client.GetChannel(channelId, "")
-	if err != nil {
+	res, resp := m.Client.GetChannelMember(channelId, m.User.Id, "")
+	if resp.Error != nil {
 		return model.GetMillis()
 	}
-	data := res.Data.(*model.ChannelData)
-	return data.Member.LastViewedAt
+	return res.LastViewedAt
 }
 
 func (m *MMClient) GetUsers() map[string]*model.User {
@@ -630,8 +728,16 @@ func (m *MMClient) GetUsers() map[string]*model.User {
 }
 
 func (m *MMClient) GetUser(userId string) *model.User {
-	m.RLock()
-	defer m.RUnlock()
+	m.Lock()
+	defer m.Unlock()
+	_, ok := m.Users[userId]
+	if !ok {
+		res, resp := m.Client.GetUser(userId, "")
+		if resp.Error != nil {
+			return nil
+		}
+		m.Users[userId] = res
+	}
 	return m.Users[userId]
 }
 
@@ -644,36 +750,36 @@ func (m *MMClient) GetUserName(userId string) string {
 }
 
 func (m *MMClient) GetStatus(userId string) string {
-	res, err := m.Client.GetStatuses()
-	if err != nil {
+	res, resp := m.Client.GetUserStatus(userId, "")
+	if resp.Error != nil {
 		return ""
 	}
-	status := res.Data.(map[string]string)
-	if status[userId] == model.STATUS_AWAY {
+	if res.Status == model.STATUS_AWAY {
 		return "away"
 	}
-	if status[userId] == model.STATUS_ONLINE {
+	if res.Status == model.STATUS_ONLINE {
 		return "online"
 	}
 	return "offline"
 }
 
 func (m *MMClient) GetStatuses() map[string]string {
-	var ok bool
+	var ids []string
 	statuses := make(map[string]string)
-	res, err := m.Client.GetStatuses()
-	if err != nil {
+	for id := range m.Users {
+		ids = append(ids, id)
+	}
+	res, resp := m.Client.GetUsersStatusesByIds(ids)
+	if resp.Error != nil {
 		return statuses
 	}
-	if statuses, ok = res.Data.(map[string]string); ok {
-		for userId, status := range statuses {
-			statuses[userId] = "offline"
-			if status == model.STATUS_AWAY {
-				statuses[userId] = "away"
-			}
-			if status == model.STATUS_ONLINE {
-				statuses[userId] = "online"
-			}
+	for _, status := range res {
+		statuses[status.UserId] = "offline"
+		if status.Status == model.STATUS_AWAY {
+			statuses[status.UserId] = "away"
+		}
+		if status.Status == model.STATUS_ONLINE {
+			statuses[status.UserId] = "online"
 		}
 	}
 	return statuses
@@ -683,7 +789,21 @@ func (m *MMClient) GetTeamId() string {
 	return m.Team.Id
 }
 
+func (m *MMClient) UploadFile(data []byte, channelId string, filename string) (string, error) {
+	f, resp := m.Client.UploadFile(data, channelId, filename)
+	if resp.Error != nil {
+		return "", resp.Error
+	}
+	return f.FileInfos[0].Id, nil
+}
+
 func (m *MMClient) StatusLoop() {
+	retries := 0
+	backoff := time.Second * 60
+	if m.OnWsConnect != nil {
+		m.OnWsConnect()
+	}
+	m.log.Debug("StatusLoop:", m.OnWsConnect)
 	for {
 		if m.WsQuit {
 			return
@@ -694,14 +814,23 @@ func (m *MMClient) StatusLoop() {
 			select {
 			case <-m.WsPingChan:
 				m.log.Debug("WS PONG received")
+				backoff = time.Second * 60
 			case <-time.After(time.Second * 5):
-				m.Logout()
-				m.WsQuit = false
-				m.Login()
-				go m.WsReceiver()
+				if retries > 3 {
+					m.Logout()
+					m.WsQuit = false
+					m.Login()
+					if m.OnWsConnect != nil {
+						m.OnWsConnect()
+					}
+					go m.WsReceiver()
+				} else {
+					retries++
+					backoff = time.Second * 5
+				}
 			}
 		}
-		time.Sleep(time.Second * 60)
+		time.Sleep(backoff)
 	}
 }
 
@@ -709,40 +838,39 @@ func (m *MMClient) StatusLoop() {
 func (m *MMClient) initUser() error {
 	m.Lock()
 	defer m.Unlock()
-	initLoad, err := m.Client.GetInitialLoad()
-	if err != nil {
-		return err
-	}
-	initData := initLoad.Data.(*model.InitialLoad)
-	m.User = initData.User
 	// we only load all team data on initial login.
 	// all other updates are for channels from our (primary) team only.
 	//m.log.Debug("initUser(): loading all team data")
-	for _, v := range initData.Teams {
-		m.Client.SetTeamId(v.Id)
-		mmusers, err := m.Client.GetProfiles(0, 50000, "")
-		if err != nil {
-			return errors.New(err.DetailedError)
+	teams, resp := m.Client.GetTeamsForUser(m.User.Id, "")
+	if resp.Error != nil {
+		return resp.Error
+	}
+	for _, team := range teams {
+		mmusers, resp := m.Client.GetUsersInTeam(team.Id, 0, 50000, "")
+		if resp.Error != nil {
+			return errors.New(resp.Error.DetailedError)
 		}
-		t := &Team{Team: v, Users: mmusers.Data.(map[string]*model.User), Id: v.Id}
-		mmchannels, err := m.Client.GetChannels("")
-		if err != nil {
-			return errors.New(err.DetailedError)
+		usermap := make(map[string]*model.User)
+		for _, user := range mmusers {
+			usermap[user.Id] = user
 		}
-		t.Channels = mmchannels.Data.(*model.ChannelList)
-		if m.mmVersion() >= 3.08 {
-			mmchannels, err = m.Client.GetMoreChannelsPage(0, 5000)
-		} else {
-			mmchannels, err = m.Client.GetMoreChannels("")
+
+		t := &Team{Team: team, Users: usermap, Id: team.Id}
+
+		mmchannels, resp := m.Client.GetChannelsForTeamForUser(team.Id, m.User.Id, "")
+		if resp.Error != nil {
+			return resp.Error
 		}
-		if err != nil {
-			return errors.New(err.DetailedError)
+		t.Channels = mmchannels
+		mmchannels, resp = m.Client.GetPublicChannelsForTeam(team.Id, 0, 5000, "")
+		if resp.Error != nil {
+			return resp.Error
 		}
-		t.MoreChannels = mmchannels.Data.(*model.ChannelList)
+		t.MoreChannels = mmchannels
 		m.OtherTeams = append(m.OtherTeams, t)
-		if v.Name == m.Credentials.Team {
+		if team.Name == m.Credentials.Team {
 			m.Team = t
-			m.log.Debugf("initUser(): found our team %s (id: %s)", v.Name, v.Id)
+			m.log.Debugf("initUser(): found our team %s (id: %s)", team.Name, team.Id)
 		}
 		// add all users
 		for k, v := range t.Users {
@@ -763,22 +891,16 @@ func (m *MMClient) sendWSRequest(action string, data map[string]interface{}) err
 	return nil
 }
 
-func (m *MMClient) mmVersion() float64 {
-	v, _ := strconv.ParseFloat(string(m.ServerVersion[0:2])+"0"+string(m.ServerVersion[2]), 64)
-	if string(m.ServerVersion[4]) == "." {
-		v, _ = strconv.ParseFloat(m.ServerVersion[0:4], 64)
-	}
-	return v
-}
-
 func supportedVersion(version string) bool {
-	if strings.HasPrefix(version, "3.5.0") ||
-		strings.HasPrefix(version, "3.6.0") ||
-		strings.HasPrefix(version, "3.7.0") ||
-		strings.HasPrefix(version, "3.8.0") ||
+	if strings.HasPrefix(version, "3.8.0") ||
 		strings.HasPrefix(version, "3.9.0") ||
-		strings.HasPrefix(version, "3.10.0") {
+		strings.HasPrefix(version, "3.10.0") ||
+		strings.HasPrefix(version, "4.") {
 		return true
 	}
 	return false
 }
+
+func digestString(s string) string {
+	return fmt.Sprintf("%x", md5.Sum([]byte(s)))
+}

matterhook/matterhook.go

@@ -17,13 +17,14 @@ import (
 
 // OMessage for mattermost incoming webhook. (send to mattermost)
 type OMessage struct {
-	Channel     string      `json:"channel,omitempty"`
-	IconURL     string      `json:"icon_url,omitempty"`
-	IconEmoji   string      `json:"icon_emoji,omitempty"`
-	UserName    string      `json:"username,omitempty"`
-	Text        string      `json:"text"`
-	Attachments interface{} `json:"attachments,omitempty"`
-	Type        string      `json:"type,omitempty"`
+	Channel     string                 `json:"channel,omitempty"`
+	IconURL     string                 `json:"icon_url,omitempty"`
+	IconEmoji   string                 `json:"icon_emoji,omitempty"`
+	UserName    string                 `json:"username,omitempty"`
+	Text        string                 `json:"text"`
+	Attachments interface{}            `json:"attachments,omitempty"`
+	Type        string                 `json:"type,omitempty"`
+	Props       map[string]interface{} `json:"props"`
 }
 
 // IMessage for mattermost outgoing webhook. (received from mattermost)
@@ -43,6 +44,7 @@ type IMessage struct {
 	ServiceId   string `schema:"service_id"`
 	Text        string `schema:"text"`
 	TriggerWord string `schema:"trigger_word"`
+	FileIDs     string `schema:"file_ids"`
 }
 
 // Client for Mattermost.
@@ -134,12 +136,11 @@ func (c *Client) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 
 // Receive returns an incoming message from mattermost outgoing webhooks URL.
 func (c *Client) Receive() IMessage {
-	for {
-		select {
-		case msg := <-c.In:
-			return msg
-		}
+	var msg IMessage
+	for msg := range c.In {
+		return msg
 	}
+	return msg
 }
 
 // Send sends a msg to mattermost incoming webhooks URL.

migration.md

@@ -1,50 +0,0 @@
-# Breaking changes from 0.4 to 0.5 for matterbridge (webhooks version)
-## IRC section
-### Server
-Port removed, added to server
-```
-server="irc.freenode.net"
-port=6667
-```
-changed to
-```
-server="irc.freenode.net:6667"
-```
-### Channel
-Removed see Channels section below
-
-### UseSlackCircumfix=true
-Removed, can be done by using ```RemoteNickFormat="<{NICK}> "```
-
-## Mattermost section
-### BindAddress
-Port removed, added to BindAddress
-
-```
-BindAddress="0.0.0.0"
-port=9999
-```
-
-changed to
-
-```
-BindAddress="0.0.0.0:9999"
-```
-
-### Token
-Removed
-
-## Channels section
-```
-[Token "outgoingwebhooktoken1"] 
-IRCChannel="#off-topic"
-MMChannel="off-topic"
-```
-
-changed to
-
-```
-[Channel "channelnameofchoice"] 
-IRC="#off-topic"
-Mattermost="off-topic"
-```

vendor/github.com/42wim/go-gitter/gitter.go

@@ -205,17 +205,43 @@ func (gitter *Gitter) GetMessage(roomID, messageID string) (*Message, error) {
 }
 
 // SendMessage sends a message to a room
-func (gitter *Gitter) SendMessage(roomID, text string) error {
+func (gitter *Gitter) SendMessage(roomID, text string) (*Message, error) {
 
 	message := Message{Text: text}
 	body, _ := json.Marshal(message)
-	_, err := gitter.post(gitter.config.apiBaseURL+"rooms/"+roomID+"/chatMessages", body)
+	response, err := gitter.post(gitter.config.apiBaseURL+"rooms/"+roomID+"/chatMessages", body)
 	if err != nil {
 		gitter.log(err)
-		return err
+		return nil, err
 	}
 
-	return nil
+	err = json.Unmarshal(response, &message)
+	if err != nil {
+		gitter.log(err)
+		return nil, err
+	}
+
+	return &message, nil
+}
+
+// UpdateMessage updates a message in a room
+func (gitter *Gitter) UpdateMessage(roomID, msgID, text string) (*Message, error) {
+
+	message := Message{Text: text}
+	body, _ := json.Marshal(message)
+	response, err := gitter.put(gitter.config.apiBaseURL+"rooms/"+roomID+"/chatMessages/"+msgID, body)
+	if err != nil {
+		gitter.log(err)
+		return nil, err
+	}
+
+	err = json.Unmarshal(response, &message)
+	if err != nil {
+		gitter.log(err)
+		return nil, err
+	}
+
+	return &message, nil
 }
 
 // JoinRoom joins a room
@@ -265,7 +291,7 @@ func (gitter *Gitter) SearchRooms(room string) ([]Room, error) {
 		Results []Room `json:"results"`
 	}
 
-	response, err := gitter.get(gitter.config.apiBaseURL + "rooms?q=" + room )
+	response, err := gitter.get(gitter.config.apiBaseURL + "rooms?q=" + room)
 
 	if err != nil {
 		gitter.log(err)
@@ -414,6 +440,39 @@ func (gitter *Gitter) post(url string, body []byte) ([]byte, error) {
 	return result, nil
 }
 
+func (gitter *Gitter) put(url string, body []byte) ([]byte, error) {
+	r, err := http.NewRequest("PUT", url, bytes.NewBuffer(body))
+	if err != nil {
+		gitter.log(err)
+		return nil, err
+	}
+
+	r.Header.Set("Content-Type", "application/json")
+	r.Header.Set("Accept", "application/json")
+	r.Header.Set("Authorization", "Bearer "+gitter.config.token)
+
+	resp, err := gitter.config.client.Do(r)
+	if err != nil {
+		gitter.log(err)
+		return nil, err
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode != http.StatusOK {
+		err = APIError{What: fmt.Sprintf("Status code: %v", resp.StatusCode)}
+		gitter.log(err)
+		return nil, err
+	}
+
+	result, err := ioutil.ReadAll(resp.Body)
+	if err != nil {
+		gitter.log(err)
+		return nil, err
+	}
+
+	return result, nil
+}
+
 func (gitter *Gitter) delete(url string) ([]byte, error) {
 	r, err := http.NewRequest("delete", url, nil)
 	if err != nil {

vendor/github.com/42wim/go-ircevent/irc.go

@@ -87,6 +87,17 @@ func (irc *Connection) readLoop() {
 	}
 }
 
+// Unescape tag values as defined in the IRCv3.2 message tags spec
+// http://ircv3.net/specs/core/message-tags-3.2.html
+func unescapeTagValue(value string) string {
+	value = strings.Replace(value, "\\:", ";", -1)
+	value = strings.Replace(value, "\\s", " ", -1)
+	value = strings.Replace(value, "\\\\", "\\", -1)
+	value = strings.Replace(value, "\\r", "\r", -1)
+	value = strings.Replace(value, "\\n", "\n", -1)
+	return value
+}
+
 //Parse raw irc messages
 func parseToEvent(msg string) (*Event, error) {
 	msg = strings.TrimSuffix(msg, "\n") //Remove \r\n
@@ -95,6 +106,26 @@ func parseToEvent(msg string) (*Event, error) {
 	if len(msg) < 5 {
 		return nil, errors.New("Malformed msg from server")
 	}
+
+	if msg[0] == '@' {
+		// IRCv3 Message Tags
+		if i := strings.Index(msg, " "); i > -1 {
+			event.Tags = make(map[string]string)
+			tags := strings.Split(msg[1:i], ";")
+			for _, data := range tags {
+				parts := strings.SplitN(data, "=", 2)
+				if len(parts) == 1 {
+					event.Tags[parts[0]] = ""
+				} else {
+					event.Tags[parts[0]] = unescapeTagValue(parts[1])
+				}
+			}
+			msg = msg[i+1 : len(msg)]
+		} else {
+			return nil, errors.New("Malformed msg from server")
+		}
+	}
+
 	if msg[0] == ':' {
 		if i := strings.Index(msg, " "); i > -1 {
 			event.Source = msg[1:i]
@@ -196,12 +227,17 @@ func (irc *Connection) isQuitting() bool {
 // Main loop to control the connection.
 func (irc *Connection) Loop() {
 	errChan := irc.ErrorChan()
+	connTime := time.Now()
 	for !irc.isQuitting() {
 		err := <-errChan
 		close(irc.end)
 		irc.Wait()
 		for !irc.isQuitting() {
 			irc.Log.Printf("Error, disconnected: %s\n", err)
+			if time.Now().Sub(connTime) < time.Second*5 {
+				irc.Log.Println("Rreconnecting too fast, sleeping 60 seconds")
+				time.Sleep(60 * time.Second)
+			}
 			if err = irc.Reconnect(); err != nil {
 				irc.Log.Printf("Error while reconnecting: %s\n", err)
 				time.Sleep(60 * time.Second)
@@ -210,6 +246,7 @@ func (irc *Connection) Loop() {
 				break
 			}
 		}
+		connTime = time.Now()
 	}
 }
 
@@ -430,26 +467,84 @@ func (irc *Connection) Connect(server string) error {
 		irc.pwrite <- fmt.Sprintf("PASS %s\r\n", irc.Password)
 	}
 
-	resChan := make(chan *SASLResult)
+	err = irc.negotiateCaps()
+	if err != nil {
+		return err
+	}
+
+	irc.pwrite <- fmt.Sprintf("NICK %s\r\n", irc.nick)
+	irc.pwrite <- fmt.Sprintf("USER %s 0.0.0.0 0.0.0.0 :%s\r\n", irc.user, irc.user)
+	return nil
+}
+
+// Negotiate IRCv3 capabilities
+func (irc *Connection) negotiateCaps() error {
+	saslResChan := make(chan *SASLResult)
+	if irc.UseSASL {
+		irc.RequestCaps = append(irc.RequestCaps, "sasl")
+		irc.setupSASLCallbacks(saslResChan)
+	}
+
+	if len(irc.RequestCaps) == 0 {
+		return nil
+	}
+
+	cap_chan := make(chan bool, len(irc.RequestCaps))
+	irc.AddCallback("CAP", func(e *Event) {
+		if len(e.Arguments) != 3 {
+			return
+		}
+		command := e.Arguments[1]
+
+		if command == "LS" {
+			missing_caps := len(irc.RequestCaps)
+			for _, cap_name := range strings.Split(e.Arguments[2], " ") {
+				for _, req_cap := range irc.RequestCaps {
+					if cap_name == req_cap {
+						irc.pwrite <- fmt.Sprintf("CAP REQ :%s\r\n", cap_name)
+						missing_caps--
+					}
+				}
+			}
+
+			for i := 0; i < missing_caps; i++ {
+				cap_chan <- true
+			}
+		} else if command == "ACK" || command == "NAK" {
+			for _, cap_name := range strings.Split(strings.TrimSpace(e.Arguments[2]), " ") {
+				if cap_name == "" {
+					continue
+				}
+
+				if command == "ACK" {
+					irc.AcknowledgedCaps = append(irc.AcknowledgedCaps, cap_name)
+				}
+				cap_chan <- true
+			}
+		}
+	})
+
+	irc.pwrite <- "CAP LS\r\n"
+
 	if irc.UseSASL {
-		irc.setupSASLCallbacks(resChan)
-		irc.pwrite <- fmt.Sprintf("CAP LS\r\n")
-		// request SASL
-		irc.pwrite <- fmt.Sprintf("CAP REQ :sasl\r\n")
-		// if sasl request doesn't complete in 15 seconds, close chan and timeout
 		select {
-		case res := <-resChan:
+		case res := <-saslResChan:
 			if res.Failed {
-				close(resChan)
+				close(saslResChan)
 				return res.Err
 			}
 		case <-time.After(time.Second * 15):
-			close(resChan)
+			close(saslResChan)
 			return errors.New("SASL setup timed out. This shouldn't happen.")
 		}
 	}
-	irc.pwrite <- fmt.Sprintf("NICK %s\r\n", irc.nick)
-	irc.pwrite <- fmt.Sprintf("USER %s 0.0.0.0 0.0.0.0 :%s\r\n", irc.user, irc.user)
+
+	// Wait for all capabilities to be ACKed or NAKed before ending negotiation
+	for i := 0; i < len(irc.RequestCaps); i++ {
+		<-cap_chan
+	}
+	irc.pwrite <- fmt.Sprintf("CAP END\r\n")
+
 	return nil
 }
 

vendor/github.com/42wim/go-ircevent/irc_sasl.go

@@ -43,7 +43,6 @@ func (irc *Connection) setupSASLCallbacks(result chan<- *SASLResult) {
 		result <- &SASLResult{true, errors.New(e.Arguments[1])}
 	})
 	irc.AddCallback("903", func(e *Event) {
-		irc.SendRaw("CAP END")
 		result <- &SASLResult{false, nil}
 	})
 	irc.AddCallback("904", func(e *Event) {

vendor/github.com/42wim/go-ircevent/irc_struct.go

@@ -15,20 +15,22 @@ import (
 type Connection struct {
 	sync.Mutex
 	sync.WaitGroup
-	Debug        bool
-	Error        chan error
-	Password     string
-	UseTLS       bool
-	UseSASL      bool
-	SASLLogin    string
-	SASLPassword string
-	SASLMech     string
-	TLSConfig    *tls.Config
-	Version      string
-	Timeout      time.Duration
-	PingFreq     time.Duration
-	KeepAlive    time.Duration
-	Server       string
+	Debug            bool
+	Error            chan error
+	Password         string
+	UseTLS           bool
+	UseSASL          bool
+	RequestCaps      []string
+	AcknowledgedCaps []string
+	SASLLogin        string
+	SASLPassword     string
+	SASLMech         string
+	TLSConfig        *tls.Config
+	Version          string
+	Timeout          time.Duration
+	PingFreq         time.Duration
+	KeepAlive        time.Duration
+	Server           string
 
 	socket net.Conn
 	pwrite chan string
@@ -59,6 +61,7 @@ type Event struct {
 	Source     string //<host>
 	User       string //<usr>
 	Arguments  []string
+	Tags       map[string]string
 	Connection *Connection
 }
 

vendor/github.com/bwmarrin/discordgo/discord.go

@@ -13,10 +13,18 @@
 // Package discordgo provides Discord binding for Go
 package discordgo
 
-import "fmt"
+import (
+	"errors"
+	"fmt"
+	"net/http"
+	"time"
+)
 
-// VERSION of Discordgo, follows Symantic Versioning. (http://semver.org/)
-const VERSION = "0.15.0"
+// VERSION of DiscordGo, follows Semantic Versioning. (http://semver.org/)
+const VERSION = "0.17.0"
+
+// ErrMFA will be risen by New when the user has 2FA.
+var ErrMFA = errors.New("account has 2FA enabled")
 
 // New creates a new Discord session and will automate some startup
 // tasks if given enough information to do so.  Currently you can pass zero
@@ -31,6 +39,12 @@ const VERSION = "0.15.0"
 //     With an email, password and auth token - Discord will verify the auth
 //         token, if it is invalid it will sign in with the provided
 //         credentials. This is the Discord recommended way to sign in.
+//
+// NOTE: While email/pass authentication is supported by DiscordGo it is
+// HIGHLY DISCOURAGED by Discord. Please only use email/pass to obtain a token
+// and then use that authentication token for all future connections.
+// Also, doing any form of automation with a user (non Bot) account may result
+// in that account being permanently banned from Discord.
 func New(args ...interface{}) (s *Session, err error) {
 
 	// Create an empty Session interface.
@@ -43,6 +57,9 @@ func New(args ...interface{}) (s *Session, err error) {
 		ShardID:                0,
 		ShardCount:             1,
 		MaxRestRetries:         3,
+		Client:                 &http.Client{Timeout: (20 * time.Second)},
+		sequence:               new(int64),
+		LastHeartbeatAck:       time.Now().UTC(),
 	}
 
 	// If no arguments are passed return the empty Session interface.
@@ -60,7 +77,7 @@ func New(args ...interface{}) (s *Session, err error) {
 
 		case []string:
 			if len(v) > 3 {
-				err = fmt.Errorf("Too many string parameters provided.")
+				err = fmt.Errorf("too many string parameters provided")
 				return
 			}
 
@@ -91,7 +108,7 @@ func New(args ...interface{}) (s *Session, err error) {
 			} else if s.Token == "" {
 				s.Token = v
 			} else {
-				err = fmt.Errorf("Too many string parameters provided.")
+				err = fmt.Errorf("too many string parameters provided")
 				return
 			}
 
@@ -99,7 +116,7 @@ func New(args ...interface{}) (s *Session, err error) {
 			// TODO: Parse configuration struct
 
 		default:
-			err = fmt.Errorf("Unsupported parameter type provided.")
+			err = fmt.Errorf("unsupported parameter type provided")
 			return
 		}
 	}
@@ -113,7 +130,11 @@ func New(args ...interface{}) (s *Session, err error) {
 	} else {
 		err = s.Login(auth, pass)
 		if err != nil || s.Token == "" {
-			err = fmt.Errorf("Unable to fetch discord authentication token. %v", err)
+			if s.MFA {
+				err = ErrMFA
+			} else {
+				err = fmt.Errorf("Unable to fetch discord authentication token. %v", err)
+			}
 			return
 		}
 	}

vendor/github.com/bwmarrin/discordgo/endpoints.go

@@ -11,6 +11,9 @@
 
 package discordgo
 
+// APIVersion is the Discord API version used for the REST and Websocket API.
+var APIVersion = "6"
+
 // Known Discord API Endpoints.
 var (
 	EndpointStatus     = "https://status.discordapp.com/api/v2/"
@@ -18,13 +21,21 @@ var (
 	EndpointSmActive   = EndpointSm + "active.json"
 	EndpointSmUpcoming = EndpointSm + "upcoming.json"
 
-	EndpointDiscord  = "https://discordapp.com/"
-	EndpointAPI      = EndpointDiscord + "api/"
-	EndpointGuilds   = EndpointAPI + "guilds/"
-	EndpointChannels = EndpointAPI + "channels/"
-	EndpointUsers    = EndpointAPI + "users/"
-	EndpointGateway  = EndpointAPI + "gateway"
-	EndpointWebhooks = EndpointAPI + "webhooks/"
+	EndpointDiscord    = "https://discordapp.com/"
+	EndpointAPI        = EndpointDiscord + "api/v" + APIVersion + "/"
+	EndpointGuilds     = EndpointAPI + "guilds/"
+	EndpointChannels   = EndpointAPI + "channels/"
+	EndpointUsers      = EndpointAPI + "users/"
+	EndpointGateway    = EndpointAPI + "gateway"
+	EndpointGatewayBot = EndpointGateway + "/bot"
+	EndpointWebhooks   = EndpointAPI + "webhooks/"
+
+	EndpointCDN             = "https://cdn.discordapp.com/"
+	EndpointCDNAttachments  = EndpointCDN + "attachments/"
+	EndpointCDNAvatars      = EndpointCDN + "avatars/"
+	EndpointCDNIcons        = EndpointCDN + "icons/"
+	EndpointCDNSplashes     = EndpointCDN + "splashes/"
+	EndpointCDNChannelIcons = EndpointCDN + "channel-icons/"
 
 	EndpointAuth           = EndpointAPI + "auth/"
 	EndpointLogin          = EndpointAuth + "login"
@@ -47,15 +58,17 @@ var (
 	EndpointReport       = EndpointAPI + "report"
 	EndpointIntegrations = EndpointAPI + "integrations"
 
-	EndpointUser              = func(uID string) string { return EndpointUsers + uID }
-	EndpointUserAvatar        = func(uID, aID string) string { return EndpointUsers + uID + "/avatars/" + aID + ".jpg" }
-	EndpointUserSettings      = func(uID string) string { return EndpointUsers + uID + "/settings" }
-	EndpointUserGuilds        = func(uID string) string { return EndpointUsers + uID + "/guilds" }
-	EndpointUserGuild         = func(uID, gID string) string { return EndpointUsers + uID + "/guilds/" + gID }
-	EndpointUserGuildSettings = func(uID, gID string) string { return EndpointUsers + uID + "/guilds/" + gID + "/settings" }
-	EndpointUserChannels      = func(uID string) string { return EndpointUsers + uID + "/channels" }
-	EndpointUserDevices       = func(uID string) string { return EndpointUsers + uID + "/devices" }
-	EndpointUserConnections   = func(uID string) string { return EndpointUsers + uID + "/connections" }
+	EndpointUser               = func(uID string) string { return EndpointUsers + uID }
+	EndpointUserAvatar         = func(uID, aID string) string { return EndpointCDNAvatars + uID + "/" + aID + ".png" }
+	EndpointUserAvatarAnimated = func(uID, aID string) string { return EndpointCDNAvatars + uID + "/" + aID + ".gif" }
+	EndpointUserSettings       = func(uID string) string { return EndpointUsers + uID + "/settings" }
+	EndpointUserGuilds         = func(uID string) string { return EndpointUsers + uID + "/guilds" }
+	EndpointUserGuild          = func(uID, gID string) string { return EndpointUsers + uID + "/guilds/" + gID }
+	EndpointUserGuildSettings  = func(uID, gID string) string { return EndpointUsers + uID + "/guilds/" + gID + "/settings" }
+	EndpointUserChannels       = func(uID string) string { return EndpointUsers + uID + "/channels" }
+	EndpointUserDevices        = func(uID string) string { return EndpointUsers + uID + "/devices" }
+	EndpointUserConnections    = func(uID string) string { return EndpointUsers + uID + "/connections" }
+	EndpointUserNotes          = func(uID string) string { return EndpointUsers + "@me/notes/" + uID }
 
 	EndpointGuild                = func(gID string) string { return EndpointGuilds + gID }
 	EndpointGuildInivtes         = func(gID string) string { return EndpointGuilds + gID + "/invites" }
@@ -73,8 +86,8 @@ var (
 	EndpointGuildInvites         = func(gID string) string { return EndpointGuilds + gID + "/invites" }
 	EndpointGuildEmbed           = func(gID string) string { return EndpointGuilds + gID + "/embed" }
 	EndpointGuildPrune           = func(gID string) string { return EndpointGuilds + gID + "/prune" }
-	EndpointGuildIcon            = func(gID, hash string) string { return EndpointGuilds + gID + "/icons/" + hash + ".jpg" }
-	EndpointGuildSplash          = func(gID, hash string) string { return EndpointGuilds + gID + "/splashes/" + hash + ".jpg" }
+	EndpointGuildIcon            = func(gID, hash string) string { return EndpointCDNIcons + gID + "/" + hash + ".png" }
+	EndpointGuildSplash          = func(gID, hash string) string { return EndpointCDNSplashes + gID + "/" + hash + ".png" }
 	EndpointGuildWebhooks        = func(gID string) string { return EndpointGuilds + gID + "/webhooks" }
 
 	EndpointChannel                   = func(cID string) string { return EndpointChannels + cID }
@@ -89,10 +102,15 @@ var (
 	EndpointChannelMessagesPins       = func(cID string) string { return EndpointChannel(cID) + "/pins" }
 	EndpointChannelMessagePin         = func(cID, mID string) string { return EndpointChannel(cID) + "/pins/" + mID }
 
+	EndpointGroupIcon = func(cID, hash string) string { return EndpointCDNChannelIcons + cID + "/" + hash + ".png" }
+
 	EndpointChannelWebhooks = func(cID string) string { return EndpointChannel(cID) + "/webhooks" }
 	EndpointWebhook         = func(wID string) string { return EndpointWebhooks + wID }
 	EndpointWebhookToken    = func(wID, token string) string { return EndpointWebhooks + wID + "/" + token }
 
+	EndpointMessageReactionsAll = func(cID, mID string) string {
+		return EndpointChannelMessage(cID, mID) + "/reactions"
+	}
 	EndpointMessageReactions = func(cID, mID, eID string) string {
 		return EndpointChannelMessage(cID, mID) + "/reactions/" + eID
 	}

vendor/github.com/bwmarrin/discordgo/event.go

@@ -1,7 +1,5 @@
 package discordgo
 
-import "fmt"
-
 // EventHandler is an interface for Discord events.
 type EventHandler interface {
 	// Type returns the type of event this handler belongs to.
@@ -45,12 +43,15 @@ var registeredInterfaceProviders = map[string]EventInterfaceProvider{}
 
 // registerInterfaceProvider registers a provider so that DiscordGo can
 // access it's New() method.
-func registerInterfaceProvider(eh EventInterfaceProvider) error {
+func registerInterfaceProvider(eh EventInterfaceProvider) {
 	if _, ok := registeredInterfaceProviders[eh.Type()]; ok {
-		return fmt.Errorf("event %s already registered", eh.Type())
+		return
+		// XXX:
+		// if we should error here, we need to do something with it.
+		// fmt.Errorf("event %s already registered", eh.Type())
 	}
 	registeredInterfaceProviders[eh.Type()] = eh
-	return nil
+	return
 }
 
 // eventHandlerInstance is a wrapper around an event handler, as functions
@@ -155,12 +156,20 @@ func (s *Session) removeEventHandlerInstance(t string, ehi *eventHandlerInstance
 // Handles calling permanent and once handlers for an event type.
 func (s *Session) handle(t string, i interface{}) {
 	for _, eh := range s.handlers[t] {
-		go eh.eventHandler.Handle(s, i)
+		if s.SyncEvents {
+			eh.eventHandler.Handle(s, i)
+		} else {
+			go eh.eventHandler.Handle(s, i)
+		}
 	}
 
 	if len(s.onceHandlers[t]) > 0 {
 		for _, eh := range s.onceHandlers[t] {
-			go eh.eventHandler.Handle(s, i)
+			if s.SyncEvents {
+				eh.eventHandler.Handle(s, i)
+			} else {
+				go eh.eventHandler.Handle(s, i)
+			}
 		}
 		s.onceHandlers[t] = nil
 	}
@@ -210,14 +219,15 @@ func (s *Session) onInterface(i interface{}) {
 		setGuildIds(t.Guild)
 	case *GuildUpdate:
 		setGuildIds(t.Guild)
-	case *Resumed:
-		s.onResumed(t)
 	case *VoiceServerUpdate:
 		go s.onVoiceServerUpdate(t)
 	case *VoiceStateUpdate:
 		go s.onVoiceStateUpdate(t)
 	}
-	s.State.onInterface(s, i)
+	err := s.State.OnInterface(s, i)
+	if err != nil {
+		s.log(LogDebug, "error dispatching internal event, %s", err)
+	}
 }
 
 // onReady handles the ready event.
@@ -225,14 +235,4 @@ func (s *Session) onReady(r *Ready) {
 
 	// Store the SessionID within the Session struct.
 	s.sessionID = r.SessionID
-
-	// Start the heartbeat to keep the connection alive.
-	go s.heartbeat(s.wsConn, s.listening, r.HeartbeatInterval)
-}
-
-// onResumed handles the resumed event.
-func (s *Session) onResumed(r *Resumed) {
-
-	// Start the heartbeat to keep the connection alive.
-	go s.heartbeat(s.wsConn, s.listening, r.HeartbeatInterval)
 }

vendor/github.com/bwmarrin/discordgo/eventhandlers.go

@@ -7,46 +7,49 @@ package discordgo
 // Event type values are used to match the events returned by Discord.
 // EventTypes surrounded by __ are synthetic and are internal to DiscordGo.
 const (
-	channelCreateEventType           = "CHANNEL_CREATE"
-	channelDeleteEventType           = "CHANNEL_DELETE"
-	channelPinsUpdateEventType       = "CHANNEL_PINS_UPDATE"
-	channelUpdateEventType           = "CHANNEL_UPDATE"
-	connectEventType                 = "__CONNECT__"
-	disconnectEventType              = "__DISCONNECT__"
-	eventEventType                   = "__EVENT__"
-	guildBanAddEventType             = "GUILD_BAN_ADD"
-	guildBanRemoveEventType          = "GUILD_BAN_REMOVE"
-	guildCreateEventType             = "GUILD_CREATE"
-	guildDeleteEventType             = "GUILD_DELETE"
-	guildEmojisUpdateEventType       = "GUILD_EMOJIS_UPDATE"
-	guildIntegrationsUpdateEventType = "GUILD_INTEGRATIONS_UPDATE"
-	guildMemberAddEventType          = "GUILD_MEMBER_ADD"
-	guildMemberRemoveEventType       = "GUILD_MEMBER_REMOVE"
-	guildMemberUpdateEventType       = "GUILD_MEMBER_UPDATE"
-	guildMembersChunkEventType       = "GUILD_MEMBERS_CHUNK"
-	guildRoleCreateEventType         = "GUILD_ROLE_CREATE"
-	guildRoleDeleteEventType         = "GUILD_ROLE_DELETE"
-	guildRoleUpdateEventType         = "GUILD_ROLE_UPDATE"
-	guildUpdateEventType             = "GUILD_UPDATE"
-	messageAckEventType              = "MESSAGE_ACK"
-	messageCreateEventType           = "MESSAGE_CREATE"
-	messageDeleteEventType           = "MESSAGE_DELETE"
-	messageReactionAddEventType      = "MESSAGE_REACTION_ADD"
-	messageReactionRemoveEventType   = "MESSAGE_REACTION_REMOVE"
-	messageUpdateEventType           = "MESSAGE_UPDATE"
-	presenceUpdateEventType          = "PRESENCE_UPDATE"
-	presencesReplaceEventType        = "PRESENCES_REPLACE"
-	rateLimitEventType               = "__RATE_LIMIT__"
-	readyEventType                   = "READY"
-	relationshipAddEventType         = "RELATIONSHIP_ADD"
-	relationshipRemoveEventType      = "RELATIONSHIP_REMOVE"
-	resumedEventType                 = "RESUMED"
-	typingStartEventType             = "TYPING_START"
-	userGuildSettingsUpdateEventType = "USER_GUILD_SETTINGS_UPDATE"
-	userSettingsUpdateEventType      = "USER_SETTINGS_UPDATE"
-	userUpdateEventType              = "USER_UPDATE"
-	voiceServerUpdateEventType       = "VOICE_SERVER_UPDATE"
-	voiceStateUpdateEventType        = "VOICE_STATE_UPDATE"
+	channelCreateEventType            = "CHANNEL_CREATE"
+	channelDeleteEventType            = "CHANNEL_DELETE"
+	channelPinsUpdateEventType        = "CHANNEL_PINS_UPDATE"
+	channelUpdateEventType            = "CHANNEL_UPDATE"
+	connectEventType                  = "__CONNECT__"
+	disconnectEventType               = "__DISCONNECT__"
+	eventEventType                    = "__EVENT__"
+	guildBanAddEventType              = "GUILD_BAN_ADD"
+	guildBanRemoveEventType           = "GUILD_BAN_REMOVE"
+	guildCreateEventType              = "GUILD_CREATE"
+	guildDeleteEventType              = "GUILD_DELETE"
+	guildEmojisUpdateEventType        = "GUILD_EMOJIS_UPDATE"
+	guildIntegrationsUpdateEventType  = "GUILD_INTEGRATIONS_UPDATE"
+	guildMemberAddEventType           = "GUILD_MEMBER_ADD"
+	guildMemberRemoveEventType        = "GUILD_MEMBER_REMOVE"
+	guildMemberUpdateEventType        = "GUILD_MEMBER_UPDATE"
+	guildMembersChunkEventType        = "GUILD_MEMBERS_CHUNK"
+	guildRoleCreateEventType          = "GUILD_ROLE_CREATE"
+	guildRoleDeleteEventType          = "GUILD_ROLE_DELETE"
+	guildRoleUpdateEventType          = "GUILD_ROLE_UPDATE"
+	guildUpdateEventType              = "GUILD_UPDATE"
+	messageAckEventType               = "MESSAGE_ACK"
+	messageCreateEventType            = "MESSAGE_CREATE"
+	messageDeleteEventType            = "MESSAGE_DELETE"
+	messageDeleteBulkEventType        = "MESSAGE_DELETE_BULK"
+	messageReactionAddEventType       = "MESSAGE_REACTION_ADD"
+	messageReactionRemoveEventType    = "MESSAGE_REACTION_REMOVE"
+	messageReactionRemoveAllEventType = "MESSAGE_REACTION_REMOVE_ALL"
+	messageUpdateEventType            = "MESSAGE_UPDATE"
+	presenceUpdateEventType           = "PRESENCE_UPDATE"
+	presencesReplaceEventType         = "PRESENCES_REPLACE"
+	rateLimitEventType                = "__RATE_LIMIT__"
+	readyEventType                    = "READY"
+	relationshipAddEventType          = "RELATIONSHIP_ADD"
+	relationshipRemoveEventType       = "RELATIONSHIP_REMOVE"
+	resumedEventType                  = "RESUMED"
+	typingStartEventType              = "TYPING_START"
+	userGuildSettingsUpdateEventType  = "USER_GUILD_SETTINGS_UPDATE"
+	userNoteUpdateEventType           = "USER_NOTE_UPDATE"
+	userSettingsUpdateEventType       = "USER_SETTINGS_UPDATE"
+	userUpdateEventType               = "USER_UPDATE"
+	voiceServerUpdateEventType        = "VOICE_SERVER_UPDATE"
+	voiceStateUpdateEventType         = "VOICE_STATE_UPDATE"
 )
 
 // channelCreateEventHandler is an event handler for ChannelCreate events.
@@ -137,11 +140,6 @@ func (eh connectEventHandler) Type() string {
 	return connectEventType
 }
 
-// New returns a new instance of Connect.
-func (eh connectEventHandler) New() interface{} {
-	return &Connect{}
-}
-
 // Handle is the handler for Connect events.
 func (eh connectEventHandler) Handle(s *Session, i interface{}) {
 	if t, ok := i.(*Connect); ok {
@@ -157,11 +155,6 @@ func (eh disconnectEventHandler) Type() string {
 	return disconnectEventType
 }
 
-// New returns a new instance of Disconnect.
-func (eh disconnectEventHandler) New() interface{} {
-	return &Disconnect{}
-}
-
 // Handle is the handler for Disconnect events.
 func (eh disconnectEventHandler) Handle(s *Session, i interface{}) {
 	if t, ok := i.(*Disconnect); ok {
@@ -177,11 +170,6 @@ func (eh eventEventHandler) Type() string {
 	return eventEventType
 }
 
-// New returns a new instance of Event.
-func (eh eventEventHandler) New() interface{} {
-	return &Event{}
-}
-
 // Handle is the handler for Event events.
 func (eh eventEventHandler) Handle(s *Session, i interface{}) {
 	if t, ok := i.(*Event); ok {
@@ -529,6 +517,26 @@ func (eh messageDeleteEventHandler) Handle(s *Session, i interface{}) {
 	}
 }
 
+// messageDeleteBulkEventHandler is an event handler for MessageDeleteBulk events.
+type messageDeleteBulkEventHandler func(*Session, *MessageDeleteBulk)
+
+// Type returns the event type for MessageDeleteBulk events.
+func (eh messageDeleteBulkEventHandler) Type() string {
+	return messageDeleteBulkEventType
+}
+
+// New returns a new instance of MessageDeleteBulk.
+func (eh messageDeleteBulkEventHandler) New() interface{} {
+	return &MessageDeleteBulk{}
+}
+
+// Handle is the handler for MessageDeleteBulk events.
+func (eh messageDeleteBulkEventHandler) Handle(s *Session, i interface{}) {
+	if t, ok := i.(*MessageDeleteBulk); ok {
+		eh(s, t)
+	}
+}
+
 // messageReactionAddEventHandler is an event handler for MessageReactionAdd events.
 type messageReactionAddEventHandler func(*Session, *MessageReactionAdd)
 
@@ -569,6 +577,26 @@ func (eh messageReactionRemoveEventHandler) Handle(s *Session, i interface{}) {
 	}
 }
 
+// messageReactionRemoveAllEventHandler is an event handler for MessageReactionRemoveAll events.
+type messageReactionRemoveAllEventHandler func(*Session, *MessageReactionRemoveAll)
+
+// Type returns the event type for MessageReactionRemoveAll events.
+func (eh messageReactionRemoveAllEventHandler) Type() string {
+	return messageReactionRemoveAllEventType
+}
+
+// New returns a new instance of MessageReactionRemoveAll.
+func (eh messageReactionRemoveAllEventHandler) New() interface{} {
+	return &MessageReactionRemoveAll{}
+}
+
+// Handle is the handler for MessageReactionRemoveAll events.
+func (eh messageReactionRemoveAllEventHandler) Handle(s *Session, i interface{}) {
+	if t, ok := i.(*MessageReactionRemoveAll); ok {
+		eh(s, t)
+	}
+}
+
 // messageUpdateEventHandler is an event handler for MessageUpdate events.
 type messageUpdateEventHandler func(*Session, *MessageUpdate)
 
@@ -637,11 +665,6 @@ func (eh rateLimitEventHandler) Type() string {
 	return rateLimitEventType
 }
 
-// New returns a new instance of RateLimit.
-func (eh rateLimitEventHandler) New() interface{} {
-	return &RateLimit{}
-}
-
 // Handle is the handler for RateLimit events.
 func (eh rateLimitEventHandler) Handle(s *Session, i interface{}) {
 	if t, ok := i.(*RateLimit); ok {
@@ -769,6 +792,26 @@ func (eh userGuildSettingsUpdateEventHandler) Handle(s *Session, i interface{})
 	}
 }
 
+// userNoteUpdateEventHandler is an event handler for UserNoteUpdate events.
+type userNoteUpdateEventHandler func(*Session, *UserNoteUpdate)
+
+// Type returns the event type for UserNoteUpdate events.
+func (eh userNoteUpdateEventHandler) Type() string {
+	return userNoteUpdateEventType
+}
+
+// New returns a new instance of UserNoteUpdate.
+func (eh userNoteUpdateEventHandler) New() interface{} {
+	return &UserNoteUpdate{}
+}
+
+// Handle is the handler for UserNoteUpdate events.
+func (eh userNoteUpdateEventHandler) Handle(s *Session, i interface{}) {
+	if t, ok := i.(*UserNoteUpdate); ok {
+		eh(s, t)
+	}
+}
+
 // userSettingsUpdateEventHandler is an event handler for UserSettingsUpdate events.
 type userSettingsUpdateEventHandler func(*Session, *UserSettingsUpdate)
 
@@ -901,10 +944,14 @@ func handlerForInterface(handler interface{}) EventHandler {
 		return messageCreateEventHandler(v)
 	case func(*Session, *MessageDelete):
 		return messageDeleteEventHandler(v)
+	case func(*Session, *MessageDeleteBulk):
+		return messageDeleteBulkEventHandler(v)
 	case func(*Session, *MessageReactionAdd):
 		return messageReactionAddEventHandler(v)
 	case func(*Session, *MessageReactionRemove):
 		return messageReactionRemoveEventHandler(v)
+	case func(*Session, *MessageReactionRemoveAll):
+		return messageReactionRemoveAllEventHandler(v)
 	case func(*Session, *MessageUpdate):
 		return messageUpdateEventHandler(v)
 	case func(*Session, *PresenceUpdate):
@@ -925,6 +972,8 @@ func handlerForInterface(handler interface{}) EventHandler {
 		return typingStartEventHandler(v)
 	case func(*Session, *UserGuildSettingsUpdate):
 		return userGuildSettingsUpdateEventHandler(v)
+	case func(*Session, *UserNoteUpdate):
+		return userNoteUpdateEventHandler(v)
 	case func(*Session, *UserSettingsUpdate):
 		return userSettingsUpdateEventHandler(v)
 	case func(*Session, *UserUpdate):
@@ -937,6 +986,7 @@ func handlerForInterface(handler interface{}) EventHandler {
 
 	return nil
 }
+
 func init() {
 	registerInterfaceProvider(channelCreateEventHandler(nil))
 	registerInterfaceProvider(channelDeleteEventHandler(nil))
@@ -959,8 +1009,10 @@ func init() {
 	registerInterfaceProvider(messageAckEventHandler(nil))
 	registerInterfaceProvider(messageCreateEventHandler(nil))
 	registerInterfaceProvider(messageDeleteEventHandler(nil))
+	registerInterfaceProvider(messageDeleteBulkEventHandler(nil))
 	registerInterfaceProvider(messageReactionAddEventHandler(nil))
 	registerInterfaceProvider(messageReactionRemoveEventHandler(nil))
+	registerInterfaceProvider(messageReactionRemoveAllEventHandler(nil))
 	registerInterfaceProvider(messageUpdateEventHandler(nil))
 	registerInterfaceProvider(presenceUpdateEventHandler(nil))
 	registerInterfaceProvider(presencesReplaceEventHandler(nil))
@@ -970,6 +1022,7 @@ func init() {
 	registerInterfaceProvider(resumedEventHandler(nil))
 	registerInterfaceProvider(typingStartEventHandler(nil))
 	registerInterfaceProvider(userGuildSettingsUpdateEventHandler(nil))
+	registerInterfaceProvider(userNoteUpdateEventHandler(nil))
 	registerInterfaceProvider(userSettingsUpdateEventHandler(nil))
 	registerInterfaceProvider(userUpdateEventHandler(nil))
 	registerInterfaceProvider(voiceServerUpdateEventHandler(nil))

vendor/github.com/bwmarrin/discordgo/events.go

@@ -2,7 +2,6 @@ package discordgo
 
 import (
 	"encoding/json"
-	"time"
 )
 
 // This file contains all the possible structs that can be
@@ -28,7 +27,7 @@ type RateLimit struct {
 // Event provides a basic initial struct for all websocket events.
 type Event struct {
 	Operation int             `json:"op"`
-	Sequence  int             `json:"s"`
+	Sequence  int64           `json:"s"`
 	Type      string          `json:"t"`
 	RawData   json.RawMessage `json:"d"`
 	// Struct contains one of the other types in this file.
@@ -37,19 +36,19 @@ type Event struct {
 
 // A Ready stores all data for the websocket READY event.
 type Ready struct {
-	Version           int           `json:"v"`
-	SessionID         string        `json:"session_id"`
-	HeartbeatInterval time.Duration `json:"heartbeat_interval"`
-	User              *User         `json:"user"`
-	ReadState         []*ReadState  `json:"read_state"`
-	PrivateChannels   []*Channel    `json:"private_channels"`
-	Guilds            []*Guild      `json:"guilds"`
+	Version         int          `json:"v"`
+	SessionID       string       `json:"session_id"`
+	User            *User        `json:"user"`
+	ReadState       []*ReadState `json:"read_state"`
+	PrivateChannels []*Channel   `json:"private_channels"`
+	Guilds          []*Guild     `json:"guilds"`
 
 	// Undocumented fields
 	Settings          *Settings            `json:"user_settings"`
 	UserGuildSettings []*UserGuildSettings `json:"user_guild_settings"`
 	Relationships     []*Relationship      `json:"relationships"`
 	Presences         []*Presence          `json:"presences"`
+	Notes             map[string]string    `json:"notes"`
 }
 
 // ChannelCreate is the data for a ChannelCreate event.
@@ -179,6 +178,11 @@ type MessageReactionRemove struct {
 	*MessageReaction
 }
 
+// MessageReactionRemoveAll is the data for a MessageReactionRemoveAll event.
+type MessageReactionRemoveAll struct {
+	*MessageReaction
+}
+
 // PresencesReplace is the data for a PresencesReplace event.
 type PresencesReplace []*Presence
 
@@ -191,8 +195,7 @@ type PresenceUpdate struct {
 
 // Resumed is the data for a Resumed event.
 type Resumed struct {
-	HeartbeatInterval time.Duration `json:"heartbeat_interval"`
-	Trace             []string      `json:"_trace"`
+	Trace []string `json:"_trace"`
 }
 
 // RelationshipAdd is the data for a RelationshipAdd event.
@@ -225,6 +228,12 @@ type UserGuildSettingsUpdate struct {
 	*UserGuildSettings
 }
 
+// UserNoteUpdate is the data for a UserNoteUpdate event.
+type UserNoteUpdate struct {
+	ID   string `json:"id"`
+	Note string `json:"note"`
+}
+
 // VoiceServerUpdate is the data for a VoiceServerUpdate event.
 type VoiceServerUpdate struct {
 	Token    string `json:"token"`
@@ -236,3 +245,9 @@ type VoiceServerUpdate struct {
 type VoiceStateUpdate struct {
 	*VoiceState
 }
+
+// MessageDeleteBulk is the data for a MessageDeleteBulk event
+type MessageDeleteBulk struct {
+	Messages  []string `json:"ids"`
+	ChannelID string   `json:"channel_id"`
+}

vendor/github.com/bwmarrin/discordgo/examples/airhorn/main.go

@@ -6,7 +6,9 @@ import (
 	"fmt"
 	"io"
 	"os"
+	"os/signal"
 	"strings"
+	"syscall"
 	"time"
 
 	"github.com/bwmarrin/discordgo"
@@ -21,6 +23,7 @@ var token string
 var buffer = make([][]byte, 0)
 
 func main() {
+
 	if token == "" {
 		fmt.Println("No token provided. Please run: airhorn -t <bot token>")
 		return
@@ -56,21 +59,37 @@ func main() {
 		fmt.Println("Error opening Discord session: ", err)
 	}
 
+	// Wait here until CTRL-C or other term signal is received.
 	fmt.Println("Airhorn is now running.  Press CTRL-C to exit.")
-	// Simple way to keep program running until CTRL-C is pressed.
-	<-make(chan struct{})
-	return
+	sc := make(chan os.Signal, 1)
+	signal.Notify(sc, syscall.SIGINT, syscall.SIGTERM, os.Interrupt, os.Kill)
+	<-sc
+
+	// Cleanly close down the Discord session.
+	dg.Close()
 }
 
+// This function will be called (due to AddHandler above) when the bot receives
+// the "ready" event from Discord.
 func ready(s *discordgo.Session, event *discordgo.Ready) {
+
 	// Set the playing status.
-	_ = s.UpdateStatus(0, "!airhorn")
+	s.UpdateStatus(0, "!airhorn")
 }
 
 // This function will be called (due to AddHandler above) every time a new
 // message is created on any channel that the autenticated bot has access to.
 func messageCreate(s *discordgo.Session, m *discordgo.MessageCreate) {
+
+	// Ignore all messages created by the bot itself
+	// This isn't required in this specific example but it's a good practice.
+	if m.Author.ID == s.State.User.ID {
+		return
+	}
+
+	// check if the message is "!airhorn"
 	if strings.HasPrefix(m.Content, "!airhorn") {
+
 		// Find the channel that the message came from.
 		c, err := s.State.Channel(m.ChannelID)
 		if err != nil {
@@ -85,7 +104,7 @@ func messageCreate(s *discordgo.Session, m *discordgo.MessageCreate) {
 			return
 		}
 
-		// Look for the message sender in that guilds current voice states.
+		// Look for the message sender in that guild's current voice states.
 		for _, vs := range g.VoiceStates {
 			if vs.UserID == m.Author.ID {
 				err = playSound(s, g.ID, vs.ChannelID)
@@ -102,6 +121,7 @@ func messageCreate(s *discordgo.Session, m *discordgo.MessageCreate) {
 // This function will be called (due to AddHandler above) every time a new
 // guild is joined.
 func guildCreate(s *discordgo.Session, event *discordgo.GuildCreate) {
+
 	if event.Guild.Unavailable {
 		return
 	}
@@ -116,8 +136,8 @@ func guildCreate(s *discordgo.Session, event *discordgo.GuildCreate) {
 
 // loadSound attempts to load an encoded sound file from disk.
 func loadSound() error {
-	file, err := os.Open("airhorn.dca")
 
+	file, err := os.Open("airhorn.dca")
 	if err != nil {
 		fmt.Println("Error opening dca file :", err)
 		return err
@@ -131,7 +151,7 @@ func loadSound() error {
 
 		// If this is the end of the file, just return.
 		if err == io.EOF || err == io.ErrUnexpectedEOF {
-			file.Close()
+			err := file.Close()
 			if err != nil {
 				return err
 			}
@@ -160,6 +180,7 @@ func loadSound() error {
 
 // playSound plays the current buffer to the provided channel.
 func playSound(s *discordgo.Session, guildID, channelID string) (err error) {
+
 	// Join the provided voice channel.
 	vc, err := s.ChannelVoiceJoin(guildID, channelID, false, true)
 	if err != nil {
@@ -170,7 +191,7 @@ func playSound(s *discordgo.Session, guildID, channelID string) (err error) {
 	time.Sleep(250 * time.Millisecond)
 
 	// Start speaking.
-	_ = vc.Speaking(true)
+	vc.Speaking(true)
 
 	// Send the buffer data.
 	for _, buff := range buffer {
@@ -178,13 +199,13 @@ func playSound(s *discordgo.Session, guildID, channelID string) (err error) {
 	}
 
 	// Stop speaking
-	_ = vc.Speaking(false)
+	vc.Speaking(false)
 
 	// Sleep for a specificed amount of time before ending.
 	time.Sleep(250 * time.Millisecond)
 
 	// Disconnect from the provided voice channel.
-	_ = vc.Disconnect()
+	vc.Disconnect()
 
 	return nil
 }

vendor/github.com/bwmarrin/discordgo/examples/appmaker/main.go

@@ -1,38 +1,42 @@
 package main
 
 import (
+	"encoding/json"
 	"flag"
 	"fmt"
+	"os"
 
 	"github.com/bwmarrin/discordgo"
 )
 
 // Variables used for command line options
 var (
-	Email    string
-	Password string
 	Token    string
-	AppName  string
+	Name     string
 	DeleteID string
 	ListOnly bool
 )
 
 func init() {
 
-	flag.StringVar(&Email, "e", "", "Account Email")
-	flag.StringVar(&Password, "p", "", "Account Password")
-	flag.StringVar(&Token, "t", "", "Account Token")
+	flag.StringVar(&Token, "t", "", "Owner Account Token")
+	flag.StringVar(&Name, "n", "", "Name to give App/Bot")
 	flag.StringVar(&DeleteID, "d", "", "Application ID to delete")
 	flag.BoolVar(&ListOnly, "l", false, "List Applications Only")
-	flag.StringVar(&AppName, "a", "", "App/Bot Name")
 	flag.Parse()
+
+	if Token == "" {
+		flag.Usage()
+		os.Exit(1)
+	}
 }
 
 func main() {
 
 	var err error
+
 	// Create a new Discord session using the provided login information.
-	dg, err := discordgo.New(Email, Password, Token)
+	dg, err := discordgo.New(Token)
 	if err != nil {
 		fmt.Println("error creating Discord session,", err)
 		return
@@ -41,18 +45,17 @@ func main() {
 	// If -l set, only display a list of existing applications
 	// for the given account.
 	if ListOnly {
-		aps, err2 := dg.Applications()
-		if err2 != nil {
+
+		aps, err := dg.Applications()
+		if err != nil {
 			fmt.Println("error fetching applications,", err)
 			return
 		}
 
-		for k, v := range aps {
-			fmt.Printf("%d : --------------------------------------\n", k)
-			fmt.Printf("ID: %s\n", v.ID)
-			fmt.Printf("Name: %s\n", v.Name)
-			fmt.Printf("Secret: %s\n", v.Secret)
-			fmt.Printf("Description: %s\n", v.Description)
+		for _, v := range aps {
+			fmt.Println("-----------------------------------------------------")
+			b, _ := json.MarshalIndent(v, "", " ")
+			fmt.Println(string(b))
 		}
 		return
 	}
@@ -66,9 +69,14 @@ func main() {
 		return
 	}
 
+	if Name == "" {
+		flag.Usage()
+		os.Exit(1)
+	}
+
 	// Create a new application.
 	ap := &discordgo.Application{}
-	ap.Name = AppName
+	ap.Name = Name
 	ap, err = dg.ApplicationCreate(ap)
 	if err != nil {
 		fmt.Println("error creating new applicaiton,", err)
@@ -76,9 +84,8 @@ func main() {
 	}
 
 	fmt.Printf("Application created successfully:\n")
-	fmt.Printf("ID: %s\n", ap.ID)
-	fmt.Printf("Name: %s\n", ap.Name)
-	fmt.Printf("Secret: %s\n\n", ap.Secret)
+	b, _ := json.MarshalIndent(ap, "", " ")
+	fmt.Println(string(b))
 
 	// Create the bot account under the application we just created
 	bot, err := dg.ApplicationBotCreate(ap.ID)
@@ -88,11 +95,9 @@ func main() {
 	}
 
 	fmt.Printf("Bot account created successfully.\n")
-	fmt.Printf("ID: %s\n", bot.ID)
-	fmt.Printf("Username: %s\n", bot.Username)
-	fmt.Printf("Token: %s\n\n", bot.Token)
+	b, _ = json.MarshalIndent(bot, "", " ")
+	fmt.Println(string(b))
+
 	fmt.Println("Please save the above posted info in a secure place.")
 	fmt.Println("You will need that information to login with your bot account.")
-
-	return
 }

vendor/github.com/bwmarrin/discordgo/examples/avatar/localfile/main.go

@@ -1,73 +0,0 @@
-package main
-
-import (
-	"encoding/base64"
-	"flag"
-	"fmt"
-	"io/ioutil"
-	"net/http"
-
-	"github.com/bwmarrin/discordgo"
-)
-
-// Variables used for command line parameters
-var (
-	Email       string
-	Password    string
-	Token       string
-	Avatar      string
-	BotID       string
-	BotUsername string
-)
-
-func init() {
-
-	flag.StringVar(&Email, "e", "", "Account Email")
-	flag.StringVar(&Password, "p", "", "Account Password")
-	flag.StringVar(&Token, "t", "", "Account Token")
-	flag.StringVar(&Avatar, "f", "./avatar.jpg", "Avatar File Name")
-	flag.Parse()
-}
-
-func main() {
-
-	// Create a new Discord session using the provided login information.
-	// Use discordgo.New(Token) to just use a token for login.
-	dg, err := discordgo.New(Email, Password, Token)
-	if err != nil {
-		fmt.Println("error creating Discord session,", err)
-		return
-	}
-
-	bot, err := dg.User("@me")
-	if err != nil {
-		fmt.Println("error fetching the bot details,", err)
-		return
-	}
-
-	BotID = bot.ID
-	BotUsername = bot.Username
-	changeAvatar(dg)
-
-	fmt.Println("Bot is now running.  Press CTRL-C to exit.")
-	// Simple way to keep program running until CTRL-C is pressed.
-	<-make(chan struct{})
-	return
-}
-
-// Helper function to change the avatar
-func changeAvatar(s *discordgo.Session) {
-	img, err := ioutil.ReadFile(Avatar)
-	if err != nil {
-		fmt.Println(err)
-	}
-
-	base64 := base64.StdEncoding.EncodeToString(img)
-
-	avatar := fmt.Sprintf("data:%s;base64,%s", http.DetectContentType(img), base64)
-
-	_, err = s.UserUpdate("", "", BotUsername, avatar, "")
-	if err != nil {
-		fmt.Println(err)
-	}
-}

vendor/github.com/bwmarrin/discordgo/examples/avatar/main.go

@@ -0,0 +1,89 @@
+package main
+
+import (
+	"encoding/base64"
+	"flag"
+	"fmt"
+	"io/ioutil"
+	"net/http"
+	"os"
+
+	"github.com/bwmarrin/discordgo"
+)
+
+// Variables used for command line parameters
+var (
+	Token      string
+	AvatarFile string
+	AvatarURL  string
+)
+
+func init() {
+
+	flag.StringVar(&Token, "t", "", "Bot Token")
+	flag.StringVar(&AvatarFile, "f", "", "Avatar File Name")
+	flag.StringVar(&AvatarURL, "u", "", "URL to the avatar image")
+	flag.Parse()
+
+	if Token == "" || (AvatarFile == "" && AvatarURL == "") {
+		flag.Usage()
+		os.Exit(1)
+	}
+}
+
+func main() {
+
+	// Create a new Discord session using the provided login information.
+	dg, err := discordgo.New("Bot " + Token)
+	if err != nil {
+		fmt.Println("error creating Discord session,", err)
+		return
+	}
+
+	// Declare these here so they can be used in the below two if blocks and
+	// still carry over to the end of this function.
+	var base64img string
+	var contentType string
+
+	// If we're using a URL link for the Avatar
+	if AvatarURL != "" {
+
+		resp, err := http.Get(AvatarURL)
+		if err != nil {
+			fmt.Println("Error retrieving the file, ", err)
+			return
+		}
+
+		defer func() {
+			_ = resp.Body.Close()
+		}()
+
+		img, err := ioutil.ReadAll(resp.Body)
+		if err != nil {
+			fmt.Println("Error reading the response, ", err)
+			return
+		}
+
+		contentType = http.DetectContentType(img)
+		base64img = base64.StdEncoding.EncodeToString(img)
+	}
+
+	// If we're using a local file for the Avatar
+	if AvatarFile != "" {
+		img, err := ioutil.ReadFile(AvatarFile)
+		if err != nil {
+			fmt.Println(err)
+		}
+
+		contentType = http.DetectContentType(img)
+		base64img = base64.StdEncoding.EncodeToString(img)
+	}
+
+	// Now lets format our base64 image into the proper format Discord wants
+	// and then call UserUpdate to set it as our user's Avatar.
+	avatar := fmt.Sprintf("data:%s;base64,%s", contentType, base64img)
+	_, err = dg.UserUpdate("", "", "", avatar, "")
+	if err != nil {
+		fmt.Println(err)
+	}
+}

vendor/github.com/bwmarrin/discordgo/examples/avatar/url/main.go

@@ -1,86 +0,0 @@
-package main
-
-import (
-	"encoding/base64"
-	"flag"
-	"fmt"
-	"io/ioutil"
-	"net/http"
-
-	"github.com/bwmarrin/discordgo"
-)
-
-// Variables used for command line parameters
-var (
-	Email       string
-	Password    string
-	Token       string
-	URL         string
-	BotID       string
-	BotUsername string
-)
-
-func init() {
-
-	flag.StringVar(&Email, "e", "", "Account Email")
-	flag.StringVar(&Password, "p", "", "Account Password")
-	flag.StringVar(&Token, "t", "", "Account Token")
-	flag.StringVar(&URL, "l", "http://bwmarrin.github.io/discordgo/img/discordgo.png", "Link to the avatar image")
-	flag.Parse()
-}
-
-func main() {
-
-	// Create a new Discord session using the provided login information.
-	// Use discordgo.New(Token) to just use a token for login.
-	dg, err := discordgo.New(Email, Password, Token)
-	if err != nil {
-		fmt.Println("error creating Discord session,", err)
-		return
-	}
-
-	bot, err := dg.User("@me")
-	if err != nil {
-		fmt.Println("error fetching the bot details,", err)
-		return
-	}
-
-	BotID = bot.ID
-	BotUsername = bot.Username
-	changeAvatar(dg)
-
-	fmt.Println("Bot is now running.  Press CTRL-C to exit.")
-	// Simple way to keep program running until CTRL-C is pressed.
-	<-make(chan struct{})
-	return
-}
-
-// Helper function to change the avatar
-func changeAvatar(s *discordgo.Session) {
-
-	resp, err := http.Get(URL)
-	if err != nil {
-		fmt.Println("Error retrieving the file, ", err)
-		return
-	}
-
-	defer func() {
-		_ = resp.Body.Close()
-	}()
-
-	img, err := ioutil.ReadAll(resp.Body)
-	if err != nil {
-		fmt.Println("Error reading the response, ", err)
-		return
-	}
-
-	base64 := base64.StdEncoding.EncodeToString(img)
-
-	avatar := fmt.Sprintf("data:%s;base64,%s", http.DetectContentType(img), base64)
-
-	_, err = s.UserUpdate("", "", BotUsername, avatar, "")
-	if err != nil {
-		fmt.Println("Error setting the avatar, ", err)
-	}
-
-}

vendor/github.com/bwmarrin/discordgo/examples/mytoken/main.go

@@ -3,6 +3,7 @@ package main
 import (
 	"flag"
 	"fmt"
+	"os"
 
 	"github.com/bwmarrin/discordgo"
 )
@@ -18,6 +19,11 @@ func init() {
 	flag.StringVar(&Email, "e", "", "Account Email")
 	flag.StringVar(&Password, "p", "", "Account Password")
 	flag.Parse()
+
+	if Email == "" || Password == "" {
+		flag.Usage()
+		os.Exit(1)
+	}
 }
 
 func main() {
@@ -29,5 +35,6 @@ func main() {
 		return
 	}
 
+	// Print out your token.
 	fmt.Printf("Your Authentication Token is:\n\n%s\n", dg.Token)
 }

vendor/github.com/bwmarrin/discordgo/examples/new_basic/main.go

@@ -1,53 +0,0 @@
-package main
-
-import (
-	"flag"
-	"fmt"
-	"time"
-
-	"github.com/bwmarrin/discordgo"
-)
-
-// Variables used for command line parameters
-var (
-	Token string
-)
-
-func init() {
-
-	flag.StringVar(&Token, "t", "", "Bot Token")
-	flag.Parse()
-}
-
-func main() {
-
-	// Create a new Discord session using the provided bot token.
-	dg, err := discordgo.New("Bot " + Token)
-	if err != nil {
-		fmt.Println("error creating Discord session,", err)
-		return
-	}
-
-	// Register messageCreate as a callback for the messageCreate events.
-	dg.AddHandler(messageCreate)
-
-	// Open the websocket and begin listening.
-	err = dg.Open()
-	if err != nil {
-		fmt.Println("error opening connection,", err)
-		return
-	}
-
-	fmt.Println("Bot is now running.  Press CTRL-C to exit.")
-	// Simple way to keep program running until CTRL-C is pressed.
-	<-make(chan struct{})
-	return
-}
-
-// This function will be called (due to AddHandler above) every time a new
-// message is created on any channel that the autenticated bot has access to.
-func messageCreate(s *discordgo.Session, m *discordgo.MessageCreate) {
-
-	// Print message to stdout.
-	fmt.Printf("%20s %20s %20s > %s\n", m.ChannelID, time.Now().Format(time.Stamp), m.Author.Username, m.Content)
-}

vendor/github.com/bwmarrin/discordgo/examples/pingpong/main.go

@@ -3,6 +3,9 @@ package main
 import (
 	"flag"
 	"fmt"
+	"os"
+	"os/signal"
+	"syscall"
 
 	"github.com/bwmarrin/discordgo"
 )
@@ -10,7 +13,6 @@ import (
 // Variables used for command line parameters
 var (
 	Token string
-	BotID string
 )
 
 func init() {
@@ -28,29 +30,24 @@ func main() {
 		return
 	}
 
-	// Get the account information.
-	u, err := dg.User("@me")
-	if err != nil {
-		fmt.Println("error obtaining account details,", err)
-	}
-
-	// Store the account ID for later use.
-	BotID = u.ID
-
-	// Register messageCreate as a callback for the messageCreate events.
+	// Register the messageCreate func as a callback for MessageCreate events.
 	dg.AddHandler(messageCreate)
 
-	// Open the websocket and begin listening.
+	// Open a websocket connection to Discord and begin listening.
 	err = dg.Open()
 	if err != nil {
 		fmt.Println("error opening connection,", err)
 		return
 	}
 
+	// Wait here until CTRL-C or other term signal is received.
 	fmt.Println("Bot is now running.  Press CTRL-C to exit.")
-	// Simple way to keep program running until CTRL-C is pressed.
-	<-make(chan struct{})
-	return
+	sc := make(chan os.Signal, 1)
+	signal.Notify(sc, syscall.SIGINT, syscall.SIGTERM, os.Interrupt, os.Kill)
+	<-sc
+
+	// Cleanly close down the Discord session.
+	dg.Close()
 }
 
 // This function will be called (due to AddHandler above) every time a new
@@ -58,17 +55,17 @@ func main() {
 func messageCreate(s *discordgo.Session, m *discordgo.MessageCreate) {
 
 	// Ignore all messages created by the bot itself
-	if m.Author.ID == BotID {
+	// This isn't required in this specific example but it's a good practice.
+	if m.Author.ID == s.State.User.ID {
 		return
 	}
-
 	// If the message is "ping" reply with "Pong!"
 	if m.Content == "ping" {
-		_, _ = s.ChannelMessageSend(m.ChannelID, "Pong!")
+		s.ChannelMessageSend(m.ChannelID, "Pong!")
 	}
 
 	// If the message is "pong" reply with "Ping!"
 	if m.Content == "pong" {
-		_, _ = s.ChannelMessageSend(m.ChannelID, "Ping!")
+		s.ChannelMessageSend(m.ChannelID, "Ping!")
 	}
 }

vendor/github.com/bwmarrin/discordgo/message.go

@@ -10,8 +10,24 @@
 package discordgo
 
 import (
-	"fmt"
+	"io"
 	"regexp"
+	"strings"
+)
+
+// MessageType is the type of Message
+type MessageType int
+
+// Block contains the valid known MessageType values
+const (
+	MessageTypeDefault MessageType = iota
+	MessageTypeRecipientAdd
+	MessageTypeRecipientRemove
+	MessageTypeCall
+	MessageTypeChannelNameChange
+	MessageTypeChannelIconChange
+	MessageTypeChannelPinnedMessage
+	MessageTypeGuildMemberJoin
 )
 
 // A Message stores all data related to a specific Discord message.
@@ -29,6 +45,58 @@ type Message struct {
 	Embeds          []*MessageEmbed      `json:"embeds"`
 	Mentions        []*User              `json:"mentions"`
 	Reactions       []*MessageReactions  `json:"reactions"`
+	Type            MessageType          `json:"type"`
+}
+
+// File stores info about files you e.g. send in messages.
+type File struct {
+	Name        string
+	ContentType string
+	Reader      io.Reader
+}
+
+// MessageSend stores all parameters you can send with ChannelMessageSendComplex.
+type MessageSend struct {
+	Content string        `json:"content,omitempty"`
+	Embed   *MessageEmbed `json:"embed,omitempty"`
+	Tts     bool          `json:"tts"`
+	Files   []*File       `json:"-"`
+
+	// TODO: Remove this when compatibility is not required.
+	File *File `json:"-"`
+}
+
+// MessageEdit is used to chain parameters via ChannelMessageEditComplex, which
+// is also where you should get the instance from.
+type MessageEdit struct {
+	Content *string       `json:"content,omitempty"`
+	Embed   *MessageEmbed `json:"embed,omitempty"`
+
+	ID      string
+	Channel string
+}
+
+// NewMessageEdit returns a MessageEdit struct, initialized
+// with the Channel and ID.
+func NewMessageEdit(channelID string, messageID string) *MessageEdit {
+	return &MessageEdit{
+		Channel: channelID,
+		ID:      messageID,
+	}
+}
+
+// SetContent is the same as setting the variable Content,
+// except it doesn't take a pointer.
+func (m *MessageEdit) SetContent(str string) *MessageEdit {
+	m.Content = &str
+	return m
+}
+
+// SetEmbed is a convenience function for setting the embed,
+// so you can chain commands.
+func (m *MessageEdit) SetEmbed(embed *MessageEmbed) *MessageEdit {
+	m.Embed = embed
+	return m
 }
 
 // A MessageAttachment stores data for message attachments.
@@ -120,13 +188,65 @@ type MessageReactions struct {
 
 // ContentWithMentionsReplaced will replace all @<id> mentions with the
 // username of the mention.
-func (m *Message) ContentWithMentionsReplaced() string {
-	if m.Mentions == nil {
-		return m.Content
-	}
-	content := m.Content
+func (m *Message) ContentWithMentionsReplaced() (content string) {
+	content = m.Content
+
 	for _, user := range m.Mentions {
-		content = regexp.MustCompile(fmt.Sprintf("<@!?(%s)>", user.ID)).ReplaceAllString(content, "@"+user.Username)
+		content = strings.NewReplacer(
+			"<@"+user.ID+">", "@"+user.Username,
+			"<@!"+user.ID+">", "@"+user.Username,
+		).Replace(content)
 	}
-	return content
+	return
+}
+
+var patternChannels = regexp.MustCompile("<#[^>]*>")
+
+// ContentWithMoreMentionsReplaced will replace all @<id> mentions with the
+// username of the mention, but also role IDs and more.
+func (m *Message) ContentWithMoreMentionsReplaced(s *Session) (content string, err error) {
+	content = m.Content
+
+	if !s.StateEnabled {
+		content = m.ContentWithMentionsReplaced()
+		return
+	}
+
+	channel, err := s.State.Channel(m.ChannelID)
+	if err != nil {
+		content = m.ContentWithMentionsReplaced()
+		return
+	}
+
+	for _, user := range m.Mentions {
+		nick := user.Username
+
+		member, err := s.State.Member(channel.GuildID, user.ID)
+		if err == nil && member.Nick != "" {
+			nick = member.Nick
+		}
+
+		content = strings.NewReplacer(
+			"<@"+user.ID+">", "@"+user.Username,
+			"<@!"+user.ID+">", "@"+nick,
+		).Replace(content)
+	}
+	for _, roleID := range m.MentionRoles {
+		role, err := s.State.Role(channel.GuildID, roleID)
+		if err != nil || !role.Mentionable {
+			continue
+		}
+
+		content = strings.Replace(content, "<&"+role.ID+">", "@"+role.Name, -1)
+	}
+
+	content = patternChannels.ReplaceAllStringFunc(content, func(mention string) string {
+		channel, err := s.State.Channel(mention[2 : len(mention)-1])
+		if err != nil || channel.Type == ChannelTypeGuildVoice {
+			return mention
+		}
+
+		return "#" + channel.Name
+	})
+	return
 }

vendor/github.com/bwmarrin/discordgo/oauth2.go

@@ -15,13 +15,18 @@ package discordgo
 
 // An Application struct stores values for a Discord OAuth2 Application
 type Application struct {
-	ID           string    `json:"id,omitempty"`
-	Name         string    `json:"name"`
-	Description  string    `json:"description,omitempty"`
-	Icon         string    `json:"icon,omitempty"`
-	Secret       string    `json:"secret,omitempty"`
-	RedirectURIs *[]string `json:"redirect_uris,omitempty"`
-	Owner        *User     `json:"owner"`
+	ID                  string    `json:"id,omitempty"`
+	Name                string    `json:"name"`
+	Description         string    `json:"description,omitempty"`
+	Icon                string    `json:"icon,omitempty"`
+	Secret              string    `json:"secret,omitempty"`
+	RedirectURIs        *[]string `json:"redirect_uris,omitempty"`
+	BotRequireCodeGrant bool      `json:"bot_require_code_grant,omitempty"`
+	BotPublic           bool      `json:"bot_public,omitempty"`
+	RPCApplicationState int       `json:"rpc_application_state,omitempty"`
+	Flags               int       `json:"flags,omitempty"`
+	Owner               *User     `json:"owner"`
+	Bot                 *User     `json:"bot"`
 }
 
 // Application returns an Application structure of a specific Application

vendor/github.com/bwmarrin/discordgo/ratelimit.go

@@ -3,16 +3,26 @@ package discordgo
 import (
 	"net/http"
 	"strconv"
+	"strings"
 	"sync"
+	"sync/atomic"
 	"time"
 )
 
+// customRateLimit holds information for defining a custom rate limit
+type customRateLimit struct {
+	suffix   string
+	requests int
+	reset    time.Duration
+}
+
 // RateLimiter holds all ratelimit buckets
 type RateLimiter struct {
 	sync.Mutex
-	global          *Bucket
-	buckets         map[string]*Bucket
-	globalRateLimit time.Duration
+	global           *int64
+	buckets          map[string]*Bucket
+	globalRateLimit  time.Duration
+	customRateLimits []*customRateLimit
 }
 
 // NewRatelimiter returns a new RateLimiter
@@ -20,7 +30,14 @@ func NewRatelimiter() *RateLimiter {
 
 	return &RateLimiter{
 		buckets: make(map[string]*Bucket),
-		global:  &Bucket{Key: "global"},
+		global:  new(int64),
+		customRateLimits: []*customRateLimit{
+			&customRateLimit{
+				suffix:   "//reactions//",
+				requests: 1,
+				reset:    200 * time.Millisecond,
+			},
+		},
 	}
 }
 
@@ -39,6 +56,14 @@ func (r *RateLimiter) getBucket(key string) *Bucket {
 		global:    r.global,
 	}
 
+	// Check if there is a custom ratelimit set for this bucket ID.
+	for _, rl := range r.customRateLimits {
+		if strings.HasSuffix(b.Key, rl.suffix) {
+			b.customRateLimit = rl
+			break
+		}
+	}
+
 	r.buckets[key] = b
 	return b
 }
@@ -58,8 +83,10 @@ func (r *RateLimiter) LockBucket(bucketID string) *Bucket {
 	}
 
 	// Check for global ratelimits
-	r.global.Lock()
-	r.global.Unlock()
+	sleepTo := time.Unix(0, atomic.LoadInt64(r.global))
+	if now := time.Now(); now.Before(sleepTo) {
+		time.Sleep(sleepTo.Sub(now))
+	}
 
 	b.remaining--
 	return b
@@ -72,14 +99,29 @@ type Bucket struct {
 	remaining int
 	limit     int
 	reset     time.Time
-	global    *Bucket
+	global    *int64
+
+	lastReset       time.Time
+	customRateLimit *customRateLimit
 }
 
 // Release unlocks the bucket and reads the headers to update the buckets ratelimit info
 // and locks up the whole thing in case if there's a global ratelimit.
 func (b *Bucket) Release(headers http.Header) error {
-
 	defer b.Unlock()
+
+	// Check if the bucket uses a custom ratelimiter
+	if rl := b.customRateLimit; rl != nil {
+		if time.Now().Sub(b.lastReset) >= rl.reset {
+			b.remaining = rl.requests - 1
+			b.lastReset = time.Now()
+		}
+		if b.remaining < 1 {
+			b.reset = time.Now().Add(rl.reset)
+		}
+		return nil
+	}
+
 	if headers == nil {
 		return nil
 	}
@@ -89,41 +131,25 @@ func (b *Bucket) Release(headers http.Header) error {
 	global := headers.Get("X-RateLimit-Global")
 	retryAfter := headers.Get("Retry-After")
 
-	// If it's global just keep the main ratelimit mutex locked
-	if global != "" {
-		parsedAfter, err := strconv.Atoi(retryAfter)
-		if err != nil {
-			return err
-		}
-
-		// Lock it in a new goroutine so that this isn't a blocking call
-		go func() {
-			// Make sure if several requests were waiting we don't sleep for n * retry-after
-			// where n is the amount of requests that were going on
-			sleepTo := time.Now().Add(time.Duration(parsedAfter) * time.Millisecond)
-
-			b.global.Lock()
-
-			sleepDuration := sleepTo.Sub(time.Now())
-			if sleepDuration > 0 {
-				time.Sleep(sleepDuration)
-			}
-
-			b.global.Unlock()
-		}()
-
-		return nil
-	}
-
-	// Update reset time if either retry after or reset headers are present
-	// Prefer retryafter because it's more accurate with time sync and whatnot
+	// Update global and per bucket reset time if the proper headers are available
+	// If global is set, then it will block all buckets until after Retry-After
+	// If Retry-After without global is provided it will use that for the new reset
+	// time since it's more accurate than X-RateLimit-Reset.
+	// If Retry-After after is not proided, it will update the reset time from X-RateLimit-Reset
 	if retryAfter != "" {
 		parsedAfter, err := strconv.ParseInt(retryAfter, 10, 64)
 		if err != nil {
 			return err
 		}
-		b.reset = time.Now().Add(time.Duration(parsedAfter) * time.Millisecond)
 
+		resetAt := time.Now().Add(time.Duration(parsedAfter) * time.Millisecond)
+
+		// Lock either this single bucket or all buckets
+		if global != "" {
+			atomic.StoreInt64(b.global, resetAt.UnixNano())
+		} else {
+			b.reset = resetAt
+		}
 	} else if reset != "" {
 		// Calculate the reset time by using the date header returned from discord
 		discordTime, err := http.ParseTime(headers.Get("Date"))

vendor/github.com/bwmarrin/discordgo/restapi.go

@@ -23,14 +23,22 @@ import (
 	"log"
 	"mime/multipart"
 	"net/http"
+	"net/textproto"
 	"net/url"
 	"strconv"
 	"strings"
 	"time"
 )
 
-// ErrJSONUnmarshal is returned for JSON Unmarshall errors.
-var ErrJSONUnmarshal = errors.New("json unmarshal")
+// All error constants
+var (
+	ErrJSONUnmarshal           = errors.New("json unmarshal")
+	ErrStatusOffline           = errors.New("You can't set your Status to offline")
+	ErrVerificationLevelBounds = errors.New("VerificationLevel out of bounds, should be between 0 and 3")
+	ErrPruneDaysBounds         = errors.New("the number of days should be more than or equal to 1")
+	ErrGuildNoIcon             = errors.New("guild does not have an icon set")
+	ErrGuildNoSplash           = errors.New("guild does not have a splash set")
+)
 
 // Request is the same as RequestWithBucketID but the bucket id is the same as the urlStr
 func (s *Session) Request(method, urlStr string, data interface{}) (response []byte, err error) {
@@ -87,9 +95,7 @@ func (s *Session) request(method, urlStr, contentType string, b []byte, bucketID
 		}
 	}
 
-	client := &http.Client{Timeout: (20 * time.Second)}
-
-	resp, err := client.Do(req)
+	resp, err := s.Client.Do(req)
 	if err != nil {
 		bucket.Release(nil)
 		return
@@ -175,6 +181,12 @@ func unmarshal(data []byte, v interface{}) error {
 // ------------------------------------------------------------------------------------------------
 
 // Login asks the Discord server for an authentication token.
+//
+// NOTE: While email/pass authentication is supported by DiscordGo it is
+// HIGHLY DISCOURAGED by Discord. Please only use email/pass to obtain a token
+// and then use that authentication token for all future connections.
+// Also, doing any form of automation with a user (non Bot) account may result
+// in that account being permanently banned from Discord.
 func (s *Session) Login(email, password string) (err error) {
 
 	data := struct {
@@ -189,6 +201,7 @@ func (s *Session) Login(email, password string) (err error) {
 
 	temp := struct {
 		Token string `json:"token"`
+		MFA   bool   `json:"mfa"`
 	}{}
 
 	err = unmarshal(response, &temp)
@@ -197,6 +210,7 @@ func (s *Session) Login(email, password string) (err error) {
 	}
 
 	s.Token = temp.Token
+	s.MFA = temp.MFA
 	return
 }
 
@@ -264,15 +278,21 @@ func (s *Session) User(userID string) (st *User, err error) {
 	return
 }
 
-// UserAvatar returns an image.Image of a users Avatar.
+// UserAvatar is deprecated. Please use UserAvatarDecode
 // userID    : A user ID or "@me" which is a shortcut of current user ID
 func (s *Session) UserAvatar(userID string) (img image.Image, err error) {
 	u, err := s.User(userID)
 	if err != nil {
 		return
 	}
+	img, err = s.UserAvatarDecode(u)
+	return
+}
 
-	body, err := s.RequestWithBucketID("GET", EndpointUserAvatar(userID, u.Avatar), nil, EndpointUserAvatar("", ""))
+// UserAvatarDecode returns an image.Image of a user's Avatar
+// user : The user which avatar should be retrieved
+func (s *Session) UserAvatarDecode(u *User) (img image.Image, err error) {
+	body, err := s.RequestWithBucketID("GET", EndpointUserAvatar(u.ID, u.Avatar), nil, EndpointUserAvatar("", ""))
 	if err != nil {
 		return
 	}
@@ -290,9 +310,9 @@ func (s *Session) UserUpdate(email, password, username, avatar, newPassword stri
 	// If left blank, avatar will be set to null/blank
 
 	data := struct {
-		Email       string `json:"email"`
-		Password    string `json:"password"`
-		Username    string `json:"username"`
+		Email       string `json:"email,omitempty"`
+		Password    string `json:"password,omitempty"`
+		Username    string `json:"username,omitempty"`
 		Avatar      string `json:"avatar,omitempty"`
 		NewPassword string `json:"new_password,omitempty"`
 	}{email, password, username, avatar, newPassword}
@@ -322,7 +342,7 @@ func (s *Session) UserSettings() (st *Settings, err error) {
 // status   : The new status (Actual valid status are 'online','idle','dnd','invisible')
 func (s *Session) UserUpdateStatus(status Status) (st *Settings, err error) {
 	if status == StatusOffline {
-		err = errors.New("You can't set your Status to offline")
+		err = ErrStatusOffline
 		return
 	}
 
@@ -370,9 +390,30 @@ func (s *Session) UserChannelCreate(recipientID string) (st *Channel, err error)
 }
 
 // UserGuilds returns an array of UserGuild structures for all guilds.
-func (s *Session) UserGuilds() (st []*UserGuild, err error) {
+// limit     : The number guilds that can be returned. (max 100)
+// beforeID  : If provided all guilds returned will be before given ID.
+// afterID   : If provided all guilds returned will be after given ID.
+func (s *Session) UserGuilds(limit int, beforeID, afterID string) (st []*UserGuild, err error) {
 
-	body, err := s.RequestWithBucketID("GET", EndpointUserGuilds("@me"), nil, EndpointUserGuilds(""))
+	v := url.Values{}
+
+	if limit > 0 {
+		v.Set("limit", strconv.Itoa(limit))
+	}
+	if afterID != "" {
+		v.Set("after", afterID)
+	}
+	if beforeID != "" {
+		v.Set("before", beforeID)
+	}
+
+	uri := EndpointUserGuilds("@me")
+
+	if len(v) > 0 {
+		uri = fmt.Sprintf("%s?%s", uri, v.Encode())
+	}
+
+	body, err := s.RequestWithBucketID("GET", uri, nil, EndpointUserGuilds(""))
 	if err != nil {
 		return
 	}
@@ -402,6 +443,13 @@ func (s *Session) UserGuildSettingsEdit(guildID string, settings *UserGuildSetti
 // NOTE: This function is now deprecated and will be removed in the future.
 // Please see the same function inside state.go
 func (s *Session) UserChannelPermissions(userID, channelID string) (apermissions int, err error) {
+	// Try to just get permissions from state.
+	apermissions, err = s.State.UserChannelPermissions(userID, channelID)
+	if err == nil {
+		return
+	}
+
+	// Otherwise try get as much data from state as possible, falling back to the network.
 	channel, err := s.State.Channel(channelID)
 	if err != nil || channel == nil {
 		channel, err = s.Channel(channelID)
@@ -431,6 +479,19 @@ func (s *Session) UserChannelPermissions(userID, channelID string) (apermissions
 		}
 	}
 
+	return memberPermissions(guild, channel, member), nil
+}
+
+// Calculates the permissions for a member.
+// https://support.discordapp.com/hc/en-us/articles/206141927-How-is-the-permission-hierarchy-structured-
+func memberPermissions(guild *Guild, channel *Channel, member *Member) (apermissions int) {
+	userID := member.User.ID
+
+	if userID == guild.OwnerID {
+		apermissions = PermissionAll
+		return
+	}
+
 	for _, role := range guild.Roles {
 		if role.ID == guild.ID {
 			apermissions |= role.Permissions
@@ -447,21 +508,36 @@ func (s *Session) UserChannelPermissions(userID, channelID string) (apermissions
 		}
 	}
 
-	if apermissions&PermissionAdministrator > 0 {
+	if apermissions&PermissionAdministrator == PermissionAdministrator {
 		apermissions |= PermissionAll
 	}
 
+	// Apply @everyone overrides from the channel.
+	for _, overwrite := range channel.PermissionOverwrites {
+		if guild.ID == overwrite.ID {
+			apermissions &= ^overwrite.Deny
+			apermissions |= overwrite.Allow
+			break
+		}
+	}
+
+	denies := 0
+	allows := 0
+
 	// Member overwrites can override role overrides, so do two passes
 	for _, overwrite := range channel.PermissionOverwrites {
 		for _, roleID := range member.Roles {
 			if overwrite.Type == "role" && roleID == overwrite.ID {
-				apermissions &= ^overwrite.Deny
-				apermissions |= overwrite.Allow
+				denies |= overwrite.Deny
+				allows |= overwrite.Allow
 				break
 			}
 		}
 	}
 
+	apermissions &= ^denies
+	apermissions |= allows
+
 	for _, overwrite := range channel.PermissionOverwrites {
 		if overwrite.Type == "member" && overwrite.ID == userID {
 			apermissions &= ^overwrite.Deny
@@ -470,11 +546,11 @@ func (s *Session) UserChannelPermissions(userID, channelID string) (apermissions
 		}
 	}
 
-	if apermissions&PermissionAdministrator > 0 {
+	if apermissions&PermissionAdministrator == PermissionAdministrator {
 		apermissions |= PermissionAllChannel
 	}
 
-	return
+	return apermissions
 }
 
 // ------------------------------------------------------------------------------------------------
@@ -527,7 +603,7 @@ func (s *Session) GuildEdit(guildID string, g GuildParams) (st *Guild, err error
 	if g.VerificationLevel != nil {
 		val := *g.VerificationLevel
 		if val < 0 || val > 3 {
-			err = errors.New("VerificationLevel out of bounds, should be between 0 and 3")
+			err = ErrVerificationLevelBounds
 			return
 		}
 	}
@@ -551,13 +627,7 @@ func (s *Session) GuildEdit(guildID string, g GuildParams) (st *Guild, err error
 		}
 	}
 
-	data := struct {
-		Name              string             `json:"name,omitempty"`
-		Region            string             `json:"region,omitempty"`
-		VerificationLevel *VerificationLevel `json:"verification_level,omitempty"`
-	}{g.Name, g.Region, g.VerificationLevel}
-
-	body, err := s.RequestWithBucketID("PATCH", EndpointGuild(guildID), data, EndpointGuild(guildID))
+	body, err := s.RequestWithBucketID("PATCH", EndpointGuild(guildID), g, EndpointGuild(guildID))
 	if err != nil {
 		return
 	}
@@ -607,11 +677,28 @@ func (s *Session) GuildBans(guildID string) (st []*GuildBan, err error) {
 // userID    : The ID of a User
 // days      : The number of days of previous comments to delete.
 func (s *Session) GuildBanCreate(guildID, userID string, days int) (err error) {
+	return s.GuildBanCreateWithReason(guildID, userID, "", days)
+}
+
+// GuildBanCreateWithReason bans the given user from the given guild also providing a reaso.
+// guildID   : The ID of a Guild.
+// userID    : The ID of a User
+// reason    : The reason for this ban
+// days      : The number of days of previous comments to delete.
+func (s *Session) GuildBanCreateWithReason(guildID, userID, reason string, days int) (err error) {
 
 	uri := EndpointGuildBan(guildID, userID)
 
+	queryParams := url.Values{}
 	if days > 0 {
-		uri = fmt.Sprintf("%s?delete-message-days=%d", uri, days)
+		queryParams.Set("delete-message-days", strconv.Itoa(days))
+	}
+	if reason != "" {
+		queryParams.Set("reason", reason)
+	}
+
+	if len(queryParams) > 0 {
+		uri += "?" + queryParams.Encode()
 	}
 
 	_, err = s.RequestWithBucketID("PUT", uri, nil, EndpointGuildBan(guildID, ""))
@@ -677,7 +764,21 @@ func (s *Session) GuildMember(guildID, userID string) (st *Member, err error) {
 // userID    : The ID of a User
 func (s *Session) GuildMemberDelete(guildID, userID string) (err error) {
 
-	_, err = s.RequestWithBucketID("DELETE", EndpointGuildMember(guildID, userID), nil, EndpointGuildMember(guildID, ""))
+	return s.GuildMemberDeleteWithReason(guildID, userID, "")
+}
+
+// GuildMemberDeleteWithReason removes the given user from the given guild.
+// guildID   : The ID of a Guild.
+// userID    : The ID of a User
+// reason    : The reason for the kick
+func (s *Session) GuildMemberDeleteWithReason(guildID, userID, reason string) (err error) {
+
+	uri := EndpointGuildMember(guildID, userID)
+	if reason != "" {
+		uri += "?reason=" + url.QueryEscape(reason)
+	}
+
+	_, err = s.RequestWithBucketID("DELETE", uri, nil, EndpointGuildMember(guildID, ""))
 	return
 }
 
@@ -722,12 +823,17 @@ func (s *Session) GuildMemberMove(guildID, userID, channelID string) (err error)
 // GuildMemberNickname updates the nickname of a guild member
 // guildID   : The ID of a guild
 // userID    : The ID of a user
+// userID    : The ID of a user or "@me" which is a shortcut of the current user ID
 func (s *Session) GuildMemberNickname(guildID, userID, nickname string) (err error) {
 
 	data := struct {
 		Nick string `json:"nick"`
 	}{nickname}
 
+	if userID == "@me" {
+		userID += "/nick"
+	}
+
 	_, err = s.RequestWithBucketID("PATCH", EndpointGuildMember(guildID, userID), data, EndpointGuildMember(guildID, ""))
 	return
 }
@@ -738,7 +844,7 @@ func (s *Session) GuildMemberNickname(guildID, userID, nickname string) (err err
 //  roleID 	  : The ID of a Role to be assigned to the user.
 func (s *Session) GuildMemberRoleAdd(guildID, userID, roleID string) (err error) {
 
-	_, err = s.RequestWithBucketID("PUT", EndpointGuildMemberRole(guildID, userID, roleID), nil, EndpointGuildMemberRole(guildID, userID, roleID))
+	_, err = s.RequestWithBucketID("PUT", EndpointGuildMemberRole(guildID, userID, roleID), nil, EndpointGuildMemberRole(guildID, "", ""))
 
 	return
 }
@@ -749,7 +855,7 @@ func (s *Session) GuildMemberRoleAdd(guildID, userID, roleID string) (err error)
 //  roleID 	  : The ID of a Role to be removed from the user.
 func (s *Session) GuildMemberRoleRemove(guildID, userID, roleID string) (err error) {
 
-	_, err = s.RequestWithBucketID("DELETE", EndpointGuildMemberRole(guildID, userID, roleID), nil, EndpointGuildMemberRole(guildID, userID, roleID))
+	_, err = s.RequestWithBucketID("DELETE", EndpointGuildMemberRole(guildID, userID, roleID), nil, EndpointGuildMemberRole(guildID, "", ""))
 
 	return
 }
@@ -904,7 +1010,7 @@ func (s *Session) GuildPruneCount(guildID string, days uint32) (count uint32, er
 	count = 0
 
 	if days <= 0 {
-		err = errors.New("The number of days should be more than or equal to 1.")
+		err = ErrPruneDaysBounds
 		return
 	}
 
@@ -934,7 +1040,7 @@ func (s *Session) GuildPrune(guildID string, days uint32) (count uint32, err err
 	count = 0
 
 	if days <= 0 {
-		err = errors.New("The number of days should be more than or equal to 1.")
+		err = ErrPruneDaysBounds
 		return
 	}
 
@@ -1036,7 +1142,7 @@ func (s *Session) GuildIcon(guildID string) (img image.Image, err error) {
 	}
 
 	if g.Icon == "" {
-		err = errors.New("Guild does not have an icon set.")
+		err = ErrGuildNoIcon
 		return
 	}
 
@@ -1058,7 +1164,7 @@ func (s *Session) GuildSplash(guildID string) (img image.Image, err error) {
 	}
 
 	if g.Splash == "" {
-		err = errors.New("Guild does not have a splash set.")
+		err = ErrGuildNoSplash
 		return
 	}
 
@@ -1156,7 +1262,8 @@ func (s *Session) ChannelTyping(channelID string) (err error) {
 // limit     : The number messages that can be returned. (max 100)
 // beforeID  : If provided all messages returned will be before given ID.
 // afterID   : If provided all messages returned will be after given ID.
-func (s *Session) ChannelMessages(channelID string, limit int, beforeID, afterID string) (st []*Message, err error) {
+// aroundID  : If provided all messages returned will be around given ID.
+func (s *Session) ChannelMessages(channelID string, limit int, beforeID, afterID, aroundID string) (st []*Message, err error) {
 
 	uri := EndpointChannelMessages(channelID)
 
@@ -1170,6 +1277,9 @@ func (s *Session) ChannelMessages(channelID string, limit int, beforeID, afterID
 	if beforeID != "" {
 		v.Set("before", beforeID)
 	}
+	if aroundID != "" {
+		v.Set("around", aroundID)
+	}
 	if len(v) > 0 {
 		uri = fmt.Sprintf("%s?%s", uri, v.Encode())
 	}
@@ -1212,20 +1322,92 @@ func (s *Session) ChannelMessageAck(channelID, messageID, lastToken string) (st
 	return
 }
 
-// channelMessageSend sends a message to the given channel.
+// ChannelMessageSend sends a message to the given channel.
 // channelID : The ID of a Channel.
 // content   : The message to send.
-// tts       : Whether to send the message with TTS.
-func (s *Session) channelMessageSend(channelID, content string, tts bool) (st *Message, err error) {
+func (s *Session) ChannelMessageSend(channelID string, content string) (*Message, error) {
+	return s.ChannelMessageSendComplex(channelID, &MessageSend{
+		Content: content,
+	})
+}
 
-	// TODO: nonce string ?
-	data := struct {
-		Content string `json:"content"`
-		TTS     bool   `json:"tts"`
-	}{content, tts}
+var quoteEscaper = strings.NewReplacer("\\", "\\\\", `"`, "\\\"")
 
-	// Send the message to the given channel
-	response, err := s.RequestWithBucketID("POST", EndpointChannelMessages(channelID), data, EndpointChannelMessages(channelID))
+// ChannelMessageSendComplex sends a message to the given channel.
+// channelID : The ID of a Channel.
+// data      : The message struct to send.
+func (s *Session) ChannelMessageSendComplex(channelID string, data *MessageSend) (st *Message, err error) {
+	if data.Embed != nil && data.Embed.Type == "" {
+		data.Embed.Type = "rich"
+	}
+
+	endpoint := EndpointChannelMessages(channelID)
+
+	// TODO: Remove this when compatibility is not required.
+	files := data.Files
+	if data.File != nil {
+		if files == nil {
+			files = []*File{data.File}
+		} else {
+			err = fmt.Errorf("cannot specify both File and Files")
+			return
+		}
+	}
+
+	var response []byte
+	if len(files) > 0 {
+		body := &bytes.Buffer{}
+		bodywriter := multipart.NewWriter(body)
+
+		var payload []byte
+		payload, err = json.Marshal(data)
+		if err != nil {
+			return
+		}
+
+		var p io.Writer
+
+		h := make(textproto.MIMEHeader)
+		h.Set("Content-Disposition", `form-data; name="payload_json"`)
+		h.Set("Content-Type", "application/json")
+
+		p, err = bodywriter.CreatePart(h)
+		if err != nil {
+			return
+		}
+
+		if _, err = p.Write(payload); err != nil {
+			return
+		}
+
+		for i, file := range files {
+			h := make(textproto.MIMEHeader)
+			h.Set("Content-Disposition", fmt.Sprintf(`form-data; name="file%d"; filename="%s"`, i, quoteEscaper.Replace(file.Name)))
+			contentType := file.ContentType
+			if contentType == "" {
+				contentType = "application/octet-stream"
+			}
+			h.Set("Content-Type", contentType)
+
+			p, err = bodywriter.CreatePart(h)
+			if err != nil {
+				return
+			}
+
+			if _, err = io.Copy(p, file.Reader); err != nil {
+				return
+			}
+		}
+
+		err = bodywriter.Close()
+		if err != nil {
+			return
+		}
+
+		response, err = s.request("POST", endpoint, bodywriter.FormDataContentType(), body.Bytes(), endpoint, 0)
+	} else {
+		response, err = s.RequestWithBucketID("POST", endpoint, data, endpoint)
+	}
 	if err != nil {
 		return
 	}
@@ -1234,55 +1416,42 @@ func (s *Session) channelMessageSend(channelID, content string, tts bool) (st *M
 	return
 }
 
-// ChannelMessageSend sends a message to the given channel.
-// channelID : The ID of a Channel.
-// content   : The message to send.
-func (s *Session) ChannelMessageSend(channelID string, content string) (st *Message, err error) {
-
-	return s.channelMessageSend(channelID, content, false)
-}
-
 // ChannelMessageSendTTS sends a message to the given channel with Text to Speech.
 // channelID : The ID of a Channel.
 // content   : The message to send.
-func (s *Session) ChannelMessageSendTTS(channelID string, content string) (st *Message, err error) {
-
-	return s.channelMessageSend(channelID, content, true)
+func (s *Session) ChannelMessageSendTTS(channelID string, content string) (*Message, error) {
+	return s.ChannelMessageSendComplex(channelID, &MessageSend{
+		Content: content,
+		Tts:     true,
+	})
 }
 
-// ChannelMessageSendEmbed sends a message to the given channel with embedded data (bot only).
+// ChannelMessageSendEmbed sends a message to the given channel with embedded data.
 // channelID : The ID of a Channel.
 // embed     : The embed data to send.
-func (s *Session) ChannelMessageSendEmbed(channelID string, embed *MessageEmbed) (st *Message, err error) {
-	if embed != nil && embed.Type == "" {
-		embed.Type = "rich"
-	}
-
-	data := struct {
-		Embed *MessageEmbed `json:"embed"`
-	}{embed}
-
-	// Send the message to the given channel
-	response, err := s.RequestWithBucketID("POST", EndpointChannelMessages(channelID), data, EndpointChannelMessages(channelID))
-	if err != nil {
-		return
-	}
-
-	err = unmarshal(response, &st)
-	return
+func (s *Session) ChannelMessageSendEmbed(channelID string, embed *MessageEmbed) (*Message, error) {
+	return s.ChannelMessageSendComplex(channelID, &MessageSend{
+		Embed: embed,
+	})
 }
 
 // ChannelMessageEdit edits an existing message, replacing it entirely with
 // the given content.
-// channeld  : The ID of a Channel
-// messageID : the ID of a Message
-func (s *Session) ChannelMessageEdit(channelID, messageID, content string) (st *Message, err error) {
+// channelID  : The ID of a Channel
+// messageID  : The ID of a Message
+// content    : The contents of the message
+func (s *Session) ChannelMessageEdit(channelID, messageID, content string) (*Message, error) {
+	return s.ChannelMessageEditComplex(NewMessageEdit(channelID, messageID).SetContent(content))
+}
 
-	data := struct {
-		Content string `json:"content"`
-	}{content}
+// ChannelMessageEditComplex edits an existing message, replacing it entirely with
+// the given MessageEdit struct
+func (s *Session) ChannelMessageEditComplex(m *MessageEdit) (st *Message, err error) {
+	if m.Embed != nil && m.Embed.Type == "" {
+		m.Embed.Type = "rich"
+	}
 
-	response, err := s.RequestWithBucketID("PATCH", EndpointChannelMessage(channelID, messageID), data, EndpointChannelMessage(channelID, ""))
+	response, err := s.RequestWithBucketID("PATCH", EndpointChannelMessage(m.Channel, m.ID), m, EndpointChannelMessage(m.Channel, ""))
 	if err != nil {
 		return
 	}
@@ -1291,26 +1460,12 @@ func (s *Session) ChannelMessageEdit(channelID, messageID, content string) (st *
 	return
 }
 
-// ChannelMessageEditEmbed edits an existing message with embedded data (bot only).
+// ChannelMessageEditEmbed edits an existing message with embedded data.
 // channelID : The ID of a Channel
 // messageID : The ID of a Message
 // embed     : The embed data to send
-func (s *Session) ChannelMessageEditEmbed(channelID, messageID string, embed *MessageEmbed) (st *Message, err error) {
-	if embed != nil && embed.Type == "" {
-		embed.Type = "rich"
-	}
-
-	data := struct {
-		Embed *MessageEmbed `json:"embed"`
-	}{embed}
-
-	response, err := s.RequestWithBucketID("PATCH", EndpointChannelMessage(channelID, messageID), data, EndpointChannelMessage(channelID, ""))
-	if err != nil {
-		return
-	}
-
-	err = unmarshal(response, &st)
-	return
+func (s *Session) ChannelMessageEditEmbed(channelID, messageID string, embed *MessageEmbed) (*Message, error) {
+	return s.ChannelMessageEditComplex(NewMessageEdit(channelID, messageID).SetEmbed(embed))
 }
 
 // ChannelMessageDelete deletes a message from the Channel.
@@ -1385,48 +1540,18 @@ func (s *Session) ChannelMessagesPinned(channelID string) (st []*Message, err er
 // channelID : The ID of a Channel.
 // name: The name of the file.
 // io.Reader : A reader for the file contents.
-func (s *Session) ChannelFileSend(channelID, name string, r io.Reader) (st *Message, err error) {
-	return s.ChannelFileSendWithMessage(channelID, "", name, r)
+func (s *Session) ChannelFileSend(channelID, name string, r io.Reader) (*Message, error) {
+	return s.ChannelMessageSendComplex(channelID, &MessageSend{File: &File{Name: name, Reader: r}})
 }
 
 // ChannelFileSendWithMessage sends a file to the given channel with an message.
+// DEPRECATED. Use ChannelMessageSendComplex instead.
 // channelID : The ID of a Channel.
 // content: Optional Message content.
 // name: The name of the file.
 // io.Reader : A reader for the file contents.
-func (s *Session) ChannelFileSendWithMessage(channelID, content string, name string, r io.Reader) (st *Message, err error) {
-
-	body := &bytes.Buffer{}
-	bodywriter := multipart.NewWriter(body)
-
-	if len(content) != 0 {
-		if err := bodywriter.WriteField("content", content); err != nil {
-			return nil, err
-		}
-	}
-
-	writer, err := bodywriter.CreateFormFile("file", name)
-	if err != nil {
-		return nil, err
-	}
-
-	_, err = io.Copy(writer, r)
-	if err != nil {
-		return
-	}
-
-	err = bodywriter.Close()
-	if err != nil {
-		return
-	}
-
-	response, err := s.request("POST", EndpointChannelMessages(channelID), bodywriter.FormDataContentType(), body.Bytes(), EndpointChannelMessages(channelID), 0)
-	if err != nil {
-		return
-	}
-
-	err = unmarshal(response, &st)
-	return
+func (s *Session) ChannelFileSendWithMessage(channelID, content string, name string, r io.Reader) (*Message, error) {
+	return s.ChannelMessageSendComplex(channelID, &MessageSend{File: &File{Name: name, Reader: r}, Content: content})
 }
 
 // ChannelInvites returns an array of Invite structures for the given channel
@@ -1563,7 +1688,7 @@ func (s *Session) VoiceICE() (st *VoiceICE, err error) {
 // Functions specific to Discord Websockets
 // ------------------------------------------------------------------------------------------------
 
-// Gateway returns the a websocket Gateway address
+// Gateway returns the websocket Gateway address
 func (s *Session) Gateway() (gateway string, err error) {
 
 	response, err := s.RequestWithBucketID("GET", EndpointGateway, nil, EndpointGateway)
@@ -1591,6 +1716,28 @@ func (s *Session) Gateway() (gateway string, err error) {
 	return
 }
 
+// GatewayBot returns the websocket Gateway address and the recommended number of shards
+func (s *Session) GatewayBot() (st *GatewayBotResponse, err error) {
+
+	response, err := s.RequestWithBucketID("GET", EndpointGatewayBot, nil, EndpointGatewayBot)
+	if err != nil {
+		return
+	}
+
+	err = unmarshal(response, &st)
+	if err != nil {
+		return
+	}
+
+	// Ensure the gateway always has a trailing slash.
+	// MacOS will fail to connect if we add query params without a trailing slash on the base domain.
+	if !strings.HasSuffix(st.URL, "/") {
+		st.URL += "/"
+	}
+
+	return
+}
+
 // Functions specific to Webhooks
 
 // WebhookCreate returns a new Webhook.
@@ -1716,14 +1863,9 @@ func (s *Session) WebhookEditWithToken(webhookID, token, name, avatar string) (s
 
 // WebhookDelete deletes a webhook for a given ID
 // webhookID: The ID of a webhook.
-func (s *Session) WebhookDelete(webhookID string) (st *Webhook, err error) {
+func (s *Session) WebhookDelete(webhookID string) (err error) {
 
-	body, err := s.RequestWithBucketID("DELETE", EndpointWebhook(webhookID), nil, EndpointWebhooks)
-	if err != nil {
-		return
-	}
-
-	err = unmarshal(body, &st)
+	_, err = s.RequestWithBucketID("DELETE", EndpointWebhook(webhookID), nil, EndpointWebhooks)
 
 	return
 }
@@ -1781,6 +1923,16 @@ func (s *Session) MessageReactionRemove(channelID, messageID, emojiID, userID st
 	return err
 }
 
+// MessageReactionsRemoveAll deletes all reactions from a message
+// channelID : The channel ID
+// messageID : The message ID.
+func (s *Session) MessageReactionsRemoveAll(channelID, messageID string) error {
+
+	_, err := s.RequestWithBucketID("DELETE", EndpointMessageReactionsAll(channelID, messageID), nil, EndpointMessageReactionsAll(channelID, messageID))
+
+	return err
+}
+
 // MessageReactions gets all the users reactions for a specific emoji.
 // channelID : The channel ID.
 // messageID : The message ID.
@@ -1808,6 +1960,20 @@ func (s *Session) MessageReactions(channelID, messageID, emojiID string, limit i
 	return
 }
 
+// ------------------------------------------------------------------------------------------------
+// Functions specific to user notes
+// ------------------------------------------------------------------------------------------------
+
+// UserNoteSet sets the note for a specific user.
+func (s *Session) UserNoteSet(userID string, message string) (err error) {
+	data := struct {
+		Note string `json:"note"`
+	}{message}
+
+	_, err = s.RequestWithBucketID("PUT", EndpointUserNotes(userID), data, EndpointUserNotes(""))
+	return
+}
+
 // ------------------------------------------------------------------------------------------------
 // Functions specific to Discord Relationships (Friends list)
 // ------------------------------------------------------------------------------------------------

vendor/github.com/bwmarrin/discordgo/state.go

@@ -14,11 +14,16 @@ package discordgo
 
 import (
 	"errors"
+	"sort"
 	"sync"
 )
 
 // ErrNilState is returned when the state is nil.
-var ErrNilState = errors.New("State not instantiated, please use discordgo.New() or assign Session.State.")
+var ErrNilState = errors.New("state not instantiated, please use discordgo.New() or assign Session.State")
+
+// ErrStateNotFound is returned when the state cache
+// requested is not found
+var ErrStateNotFound = errors.New("state cache not found")
 
 // A State contains the current known state.
 // As discord sends this in a READY blob, it seems reasonable to simply
@@ -33,9 +38,11 @@ type State struct {
 	TrackMembers    bool
 	TrackRoles      bool
 	TrackVoice      bool
+	TrackPresences  bool
 
 	guildMap   map[string]*Guild
 	channelMap map[string]*Channel
+	memberMap  map[string]map[string]*Member
 }
 
 // NewState creates an empty state.
@@ -45,16 +52,26 @@ func NewState() *State {
 			PrivateChannels: []*Channel{},
 			Guilds:          []*Guild{},
 		},
-		TrackChannels: true,
-		TrackEmojis:   true,
-		TrackMembers:  true,
-		TrackRoles:    true,
-		TrackVoice:    true,
-		guildMap:      make(map[string]*Guild),
-		channelMap:    make(map[string]*Channel),
+		TrackChannels:  true,
+		TrackEmojis:    true,
+		TrackMembers:   true,
+		TrackRoles:     true,
+		TrackVoice:     true,
+		TrackPresences: true,
+		guildMap:       make(map[string]*Guild),
+		channelMap:     make(map[string]*Channel),
+		memberMap:      make(map[string]map[string]*Member),
 	}
 }
 
+func (s *State) createMemberMap(guild *Guild) {
+	members := make(map[string]*Member)
+	for _, m := range guild.Members {
+		members[m.User.ID] = m
+	}
+	s.memberMap[guild.ID] = members
+}
+
 // GuildAdd adds a guild to the current world state, or
 // updates it if it already exists.
 func (s *State) GuildAdd(guild *Guild) error {
@@ -70,6 +87,14 @@ func (s *State) GuildAdd(guild *Guild) error {
 		s.channelMap[c.ID] = c
 	}
 
+	// If this guild contains a new member slice, we must regenerate the member map so the pointers stay valid
+	if guild.Members != nil {
+		s.createMemberMap(guild)
+	} else if _, ok := s.memberMap[guild.ID]; !ok {
+		// Even if we have no new member slice, we still initialize the member map for this guild if it doesn't exist
+		s.memberMap[guild.ID] = make(map[string]*Member)
+	}
+
 	if g, ok := s.guildMap[guild.ID]; ok {
 		// We are about to replace `g` in the state with `guild`, but first we need to
 		// make sure we preserve any fields that the `guild` doesn't contain from `g`.
@@ -143,7 +168,108 @@ func (s *State) Guild(guildID string) (*Guild, error) {
 		return g, nil
 	}
 
-	return nil, errors.New("Guild not found.")
+	return nil, ErrStateNotFound
+}
+
+// PresenceAdd adds a presence to the current world state, or
+// updates it if it already exists.
+func (s *State) PresenceAdd(guildID string, presence *Presence) error {
+	if s == nil {
+		return ErrNilState
+	}
+
+	guild, err := s.Guild(guildID)
+	if err != nil {
+		return err
+	}
+
+	s.Lock()
+	defer s.Unlock()
+
+	for i, p := range guild.Presences {
+		if p.User.ID == presence.User.ID {
+			//guild.Presences[i] = presence
+
+			//Update status
+			guild.Presences[i].Game = presence.Game
+			guild.Presences[i].Roles = presence.Roles
+			if presence.Status != "" {
+				guild.Presences[i].Status = presence.Status
+			}
+			if presence.Nick != "" {
+				guild.Presences[i].Nick = presence.Nick
+			}
+
+			//Update the optionally sent user information
+			//ID Is a mandatory field so you should not need to check if it is empty
+			guild.Presences[i].User.ID = presence.User.ID
+
+			if presence.User.Avatar != "" {
+				guild.Presences[i].User.Avatar = presence.User.Avatar
+			}
+			if presence.User.Discriminator != "" {
+				guild.Presences[i].User.Discriminator = presence.User.Discriminator
+			}
+			if presence.User.Email != "" {
+				guild.Presences[i].User.Email = presence.User.Email
+			}
+			if presence.User.Token != "" {
+				guild.Presences[i].User.Token = presence.User.Token
+			}
+			if presence.User.Username != "" {
+				guild.Presences[i].User.Username = presence.User.Username
+			}
+
+			return nil
+		}
+	}
+
+	guild.Presences = append(guild.Presences, presence)
+	return nil
+}
+
+// PresenceRemove removes a presence from the current world state.
+func (s *State) PresenceRemove(guildID string, presence *Presence) error {
+	if s == nil {
+		return ErrNilState
+	}
+
+	guild, err := s.Guild(guildID)
+	if err != nil {
+		return err
+	}
+
+	s.Lock()
+	defer s.Unlock()
+
+	for i, p := range guild.Presences {
+		if p.User.ID == presence.User.ID {
+			guild.Presences = append(guild.Presences[:i], guild.Presences[i+1:]...)
+			return nil
+		}
+	}
+
+	return ErrStateNotFound
+}
+
+// Presence gets a presence by ID from a guild.
+func (s *State) Presence(guildID, userID string) (*Presence, error) {
+	if s == nil {
+		return nil, ErrNilState
+	}
+
+	guild, err := s.Guild(guildID)
+	if err != nil {
+		return nil, err
+	}
+
+	for _, p := range guild.Presences {
+		if p.User.ID == userID {
+			return p, nil
+		}
+	}
+
+	return nil, ErrStateNotFound
 }
 
 // TODO: Consider moving Guild state update methods onto *Guild.
@@ -163,14 +289,19 @@ func (s *State) MemberAdd(member *Member) error {
 	s.Lock()
 	defer s.Unlock()
 
-	for i, m := range guild.Members {
-		if m.User.ID == member.User.ID {
-			guild.Members[i] = member
-			return nil
-		}
+	members, ok := s.memberMap[member.GuildID]
+	if !ok {
+		return ErrStateNotFound
+	}
+
+	m, ok := members[member.User.ID]
+	if !ok {
+		members[member.User.ID] = member
+		guild.Members = append(guild.Members, member)
+	} else {
+		*m = *member // Update the actual data, which will also update the member pointer in the slice
 	}
 
-	guild.Members = append(guild.Members, member)
 	return nil
 }
 
@@ -188,6 +319,17 @@ func (s *State) MemberRemove(member *Member) error {
 	s.Lock()
 	defer s.Unlock()
 
+	members, ok := s.memberMap[member.GuildID]
+	if !ok {
+		return ErrStateNotFound
+	}
+
+	_, ok = members[member.User.ID]
+	if !ok {
+		return ErrStateNotFound
+	}
+	delete(members, member.User.ID)
+
 	for i, m := range guild.Members {
 		if m.User.ID == member.User.ID {
 			guild.Members = append(guild.Members[:i], guild.Members[i+1:]...)
@@ -195,7 +337,7 @@ func (s *State) MemberRemove(member *Member) error {
 		}
 	}
 
-	return errors.New("Member not found.")
+	return ErrStateNotFound
 }
 
 // Member gets a member by ID from a guild.
@@ -204,21 +346,20 @@ func (s *State) Member(guildID, userID string) (*Member, error) {
 		return nil, ErrNilState
 	}
 
-	guild, err := s.Guild(guildID)
-	if err != nil {
-		return nil, err
-	}
-
 	s.RLock()
 	defer s.RUnlock()
 
-	for _, m := range guild.Members {
-		if m.User.ID == userID {
-			return m, nil
-		}
+	members, ok := s.memberMap[guildID]
+	if !ok {
+		return nil, ErrStateNotFound
 	}
 
-	return nil, errors.New("Member not found.")
+	m, ok := members[userID]
+	if ok {
+		return m, nil
+	}
+
+	return nil, ErrStateNotFound
 }
 
 // RoleAdd adds a role to the current world state, or
@@ -268,7 +409,7 @@ func (s *State) RoleRemove(guildID, roleID string) error {
 		}
 	}
 
-	return errors.New("Role not found.")
+	return ErrStateNotFound
 }
 
 // Role gets a role by ID from a guild.
@@ -291,10 +432,10 @@ func (s *State) Role(guildID, roleID string) (*Role, error) {
 		}
 	}
 
-	return nil, errors.New("Role not found.")
+	return nil, ErrStateNotFound
 }
 
-// ChannelAdd adds a guild to the current world state, or
+// ChannelAdd adds a channel to the current world state, or
 // updates it if it already exists.
 // Channels may exist either as PrivateChannels or inside
 // a guild.
@@ -319,12 +460,12 @@ func (s *State) ChannelAdd(channel *Channel) error {
 		return nil
 	}
 
-	if channel.IsPrivate {
+	if channel.Type == ChannelTypeDM || channel.Type == ChannelTypeGroupDM {
 		s.PrivateChannels = append(s.PrivateChannels, channel)
 	} else {
 		guild, ok := s.guildMap[channel.GuildID]
 		if !ok {
-			return errors.New("Guild for channel not found.")
+			return ErrStateNotFound
 		}
 
 		guild.Channels = append(guild.Channels, channel)
@@ -346,7 +487,7 @@ func (s *State) ChannelRemove(channel *Channel) error {
 		return err
 	}
 
-	if channel.IsPrivate {
+	if channel.Type == ChannelTypeDM || channel.Type == ChannelTypeGroupDM {
 		s.Lock()
 		defer s.Unlock()
 
@@ -403,7 +544,7 @@ func (s *State) Channel(channelID string) (*Channel, error) {
 		return c, nil
 	}
 
-	return nil, errors.New("Channel not found.")
+	return nil, ErrStateNotFound
 }
 
 // Emoji returns an emoji for a guild and emoji id.
@@ -426,7 +567,7 @@ func (s *State) Emoji(guildID, emojiID string) (*Emoji, error) {
 		}
 	}
 
-	return nil, errors.New("Emoji not found.")
+	return nil, ErrStateNotFound
 }
 
 // EmojiAdd adds an emoji to the current world state.
@@ -523,7 +664,12 @@ func (s *State) MessageRemove(message *Message) error {
 		return ErrNilState
 	}
 
-	c, err := s.Channel(message.ChannelID)
+	return s.messageRemoveByID(message.ChannelID, message.ID)
+}
+
+// messageRemoveByID removes a message by channelID and messageID from the world state.
+func (s *State) messageRemoveByID(channelID, messageID string) error {
+	c, err := s.Channel(channelID)
 	if err != nil {
 		return err
 	}
@@ -532,13 +678,13 @@ func (s *State) MessageRemove(message *Message) error {
 	defer s.Unlock()
 
 	for i, m := range c.Messages {
-		if m.ID == message.ID {
+		if m.ID == messageID {
 			c.Messages = append(c.Messages[:i], c.Messages[i+1:]...)
 			return nil
 		}
 	}
 
-	return errors.New("Message not found.")
+	return ErrStateNotFound
 }
 
 func (s *State) voiceStateUpdate(update *VoiceStateUpdate) error {
@@ -592,7 +738,7 @@ func (s *State) Message(channelID, messageID string) (*Message, error) {
 		}
 	}
 
-	return nil, errors.New("Message not found.")
+	return nil, ErrStateNotFound
 }
 
 // OnReady takes a Ready event and updates all internal state.
@@ -608,10 +754,9 @@ func (s *State) onReady(se *Session, r *Ready) (err error) {
 	// if state is disabled, store the bare essentials.
 	if !se.StateEnabled {
 		ready := Ready{
-			Version:           r.Version,
-			SessionID:         r.SessionID,
-			HeartbeatInterval: r.HeartbeatInterval,
-			User:              r.User,
+			Version:   r.Version,
+			SessionID: r.SessionID,
+			User:      r.User,
 		}
 
 		s.Ready = ready
@@ -623,6 +768,7 @@ func (s *State) onReady(se *Session, r *Ready) (err error) {
 
 	for _, g := range s.Guilds {
 		s.guildMap[g.ID] = g
+		s.createMemberMap(g)
 
 		for _, c := range g.Channels {
 			s.channelMap[c.ID] = c
@@ -636,8 +782,8 @@ func (s *State) onReady(se *Session, r *Ready) (err error) {
 	return nil
 }
 
-// onInterface handles all events related to states.
-func (s *State) onInterface(se *Session, i interface{}) (err error) {
+// OnInterface handles all events related to states.
+func (s *State) OnInterface(se *Session, i interface{}) (err error) {
 	if s == nil {
 		return ErrNilState
 	}
@@ -710,10 +856,55 @@ func (s *State) onInterface(se *Session, i interface{}) (err error) {
 		if s.MaxMessageCount != 0 {
 			err = s.MessageRemove(t.Message)
 		}
+	case *MessageDeleteBulk:
+		if s.MaxMessageCount != 0 {
+			for _, mID := range t.Messages {
+				s.messageRemoveByID(t.ChannelID, mID)
+			}
+		}
 	case *VoiceStateUpdate:
 		if s.TrackVoice {
 			err = s.voiceStateUpdate(t)
 		}
+	case *PresenceUpdate:
+		if s.TrackPresences {
+			s.PresenceAdd(t.GuildID, &t.Presence)
+		}
+		if s.TrackMembers {
+			if t.Status == StatusOffline {
+				return
+			}
+
+			var m *Member
+			m, err = s.Member(t.GuildID, t.User.ID)
+
+			if err != nil {
+				// Member not found; this is a user coming online
+				m = &Member{
+					GuildID: t.GuildID,
+					Nick:    t.Nick,
+					User:    t.User,
+					Roles:   t.Roles,
+				}
+
+			} else {
+
+				if t.Nick != "" {
+					m.Nick = t.Nick
+				}
+
+				if t.User.Username != "" {
+					m.User.Username = t.User.Username
+				}
+
+				// PresenceUpdates always contain a list of roles, so there's no need to check for an empty list here
+				m.Roles = t.Roles
+
+			}
+
+			err = s.MemberAdd(m)
+		}
+
 	}
 
 	return
@@ -747,48 +938,46 @@ func (s *State) UserChannelPermissions(userID, channelID string) (apermissions i
 		return
 	}
 
-	for _, role := range guild.Roles {
-		if role.ID == guild.ID {
-			apermissions |= role.Permissions
-			break
-		}
+	return memberPermissions(guild, channel, member), nil
+}
+
+// UserColor returns the color of a user in a channel.
+// While colors are defined at a Guild level, determining for a channel is more useful in message handlers.
+// 0 is returned in cases of error, which is the color of @everyone.
+// userID    : The ID of the user to calculate the color for.
+// channelID   : The ID of the channel to calculate the color for.
+func (s *State) UserColor(userID, channelID string) int {
+	if s == nil {
+		return 0
 	}
 
-	for _, role := range guild.Roles {
+	channel, err := s.Channel(channelID)
+	if err != nil {
+		return 0
+	}
+
+	guild, err := s.Guild(channel.GuildID)
+	if err != nil {
+		return 0
+	}
+
+	member, err := s.Member(guild.ID, userID)
+	if err != nil {
+		return 0
+	}
+
+	roles := Roles(guild.Roles)
+	sort.Sort(roles)
+
+	for _, role := range roles {
 		for _, roleID := range member.Roles {
 			if role.ID == roleID {
-				apermissions |= role.Permissions
-				break
+				if role.Color != 0 {
+					return role.Color
+				}
 			}
 		}
 	}
 
-	if apermissions&PermissionAdministrator > 0 {
-		apermissions |= PermissionAll
-	}
-
-	// Member overwrites can override role overrides, so do two passes
-	for _, overwrite := range channel.PermissionOverwrites {
-		for _, roleID := range member.Roles {
-			if overwrite.Type == "role" && roleID == overwrite.ID {
-				apermissions &= ^overwrite.Deny
-				apermissions |= overwrite.Allow
-				break
-			}
-		}
-	}
-
-	for _, overwrite := range channel.PermissionOverwrites {
-		if overwrite.Type == "member" && overwrite.ID == userID {
-			apermissions &= ^overwrite.Deny
-			apermissions |= overwrite.Allow
-			break
-		}
-	}
-
-	if apermissions&PermissionAdministrator > 0 {
-		apermissions |= PermissionAllChannel
-	}
-
-	return
+	return 0
 }

vendor/github.com/bwmarrin/discordgo/structs.go

@@ -13,6 +13,7 @@ package discordgo
 
 import (
 	"encoding/json"
+	"net/http"
 	"strconv"
 	"sync"
 	"time"
@@ -28,6 +29,7 @@ type Session struct {
 
 	// Authentication token for this session
 	Token string
+	MFA   bool
 
 	// Debug for printing JSON request/responses
 	Debug    bool // Deprecated, will be removed.
@@ -48,6 +50,10 @@ type Session struct {
 	// active guilds and the members of the guilds.
 	StateEnabled bool
 
+	// Whether or not to call event handlers synchronously.
+	// e.g false = launch event handlers in their own goroutines.
+	SyncEvents bool
+
 	// Exposed but should not be modified by User.
 
 	// Whether the Data Websocket is ready
@@ -73,6 +79,12 @@ type Session struct {
 	// StateEnabled is true.
 	State *State
 
+	// The http client used for REST requests
+	Client *http.Client
+
+	// Stores the last HeartbeatAck that was recieved (in UTC)
+	LastHeartbeatAck time.Time
+
 	// Event handlers
 	handlersMu   sync.RWMutex
 	handlers     map[string][]*eventHandlerInstance
@@ -88,7 +100,7 @@ type Session struct {
 	ratelimiter *RateLimiter
 
 	// sequence tracks the current gateway api websocket sequence number
-	sequence int
+	sequence *int64
 
 	// stores sessions current Discord Gateway
 	gateway string
@@ -100,12 +112,6 @@ type Session struct {
 	wsMutex sync.Mutex
 }
 
-type rateLimitMutex struct {
-	sync.Mutex
-	url map[string]*sync.Mutex
-	// bucket map[string]*sync.Mutex // TODO :)
-}
-
 // A VoiceRegion stores data for a specific voice region server.
 type VoiceRegion struct {
 	ID       string `json:"id"`
@@ -142,18 +148,30 @@ type Invite struct {
 	Temporary bool      `json:"temporary"`
 }
 
+// ChannelType is the type of a Channel
+type ChannelType int
+
+// Block contains known ChannelType values
+const (
+	ChannelTypeGuildText ChannelType = iota
+	ChannelTypeDM
+	ChannelTypeGuildVoice
+	ChannelTypeGroupDM
+	ChannelTypeGuildCategory
+)
+
 // A Channel holds all data related to an individual Discord channel.
 type Channel struct {
 	ID                   string                 `json:"id"`
 	GuildID              string                 `json:"guild_id"`
 	Name                 string                 `json:"name"`
 	Topic                string                 `json:"topic"`
-	Type                 string                 `json:"type"`
+	Type                 ChannelType            `json:"type"`
 	LastMessageID        string                 `json:"last_message_id"`
+	NSFW                 bool                   `json:"nsfw"`
 	Position             int                    `json:"position"`
 	Bitrate              int                    `json:"bitrate"`
-	IsPrivate            bool                   `json:"is_private"`
-	Recipient            *User                  `json:"recipient"`
+	Recipients           []*User                `json:"recipient"`
 	Messages             []*Message             `json:"-"`
 	PermissionOverwrites []*PermissionOverwrite `json:"permission_overwrites"`
 }
@@ -235,9 +253,15 @@ type UserGuild struct {
 
 // A GuildParams stores all the data needed to update discord guild settings
 type GuildParams struct {
-	Name              string             `json:"name"`
-	Region            string             `json:"region"`
-	VerificationLevel *VerificationLevel `json:"verification_level"`
+	Name                        string             `json:"name,omitempty"`
+	Region                      string             `json:"region,omitempty"`
+	VerificationLevel           *VerificationLevel `json:"verification_level,omitempty"`
+	DefaultMessageNotifications int                `json:"default_message_notifications,omitempty"` // TODO: Separate type?
+	AfkChannelID                string             `json:"afk_channel_id,omitempty"`
+	AfkTimeout                  int                `json:"afk_timeout,omitempty"`
+	Icon                        string             `json:"icon,omitempty"`
+	OwnerID                     string             `json:"owner_id,omitempty"`
+	Splash                      string             `json:"splash,omitempty"`
 }
 
 // A Role stores information about Discord guild member roles.
@@ -252,6 +276,21 @@ type Role struct {
 	Permissions int    `json:"permissions"`
 }
 
+// Roles are a collection of Role
+type Roles []*Role
+
+func (r Roles) Len() int {
+	return len(r)
+}
+
+func (r Roles) Less(i, j int) bool {
+	return r[i].Position > r[j].Position
+}
+
+func (r Roles) Swap(i, j int) {
+	r[i], r[j] = r[j], r[i]
+}
+
 // A VoiceState stores the voice states of Guilds
 type VoiceState struct {
 	UserID    string `json:"user_id"`
@@ -272,19 +311,20 @@ type Presence struct {
 	Game   *Game    `json:"game"`
 	Nick   string   `json:"nick"`
 	Roles  []string `json:"roles"`
+	Since  *int     `json:"since"`
 }
 
 // A Game struct holds the name of the "playing .." game for a user
 type Game struct {
 	Name string `json:"name"`
 	Type int    `json:"type"`
-	URL  string `json:"url"`
+	URL  string `json:"url,omitempty"`
 }
 
 // UnmarshalJSON unmarshals json to Game struct
 func (g *Game) UnmarshalJSON(bytes []byte) error {
 	temp := &struct {
-		Name string          `json:"name"`
+		Name json.Number     `json:"name"`
 		Type json.RawMessage `json:"type"`
 		URL  string          `json:"url"`
 	}{}
@@ -292,8 +332,8 @@ func (g *Game) UnmarshalJSON(bytes []byte) error {
 	if err != nil {
 		return err
 	}
-	g.Name = temp.Name
 	g.URL = temp.URL
+	g.Name = temp.Name.String()
 
 	if temp.Type != nil {
 		err = json.Unmarshal(temp.Type, &g.Type)
@@ -324,19 +364,6 @@ type Member struct {
 	Roles    []string `json:"roles"`
 }
 
-// A User stores all data for an individual Discord user.
-type User struct {
-	ID            string `json:"id"`
-	Email         string `json:"email"`
-	Username      string `json:"username"`
-	Avatar        string `json:"Avatar"`
-	Discriminator string `json:"discriminator"`
-	Token         string `json:"token"`
-	Verified      bool   `json:"verified"`
-	MFAEnabled    bool   `json:"mfa_enabled"`
-	Bot           bool   `json:"bot"`
-}
-
 // A Settings stores data for a specific users Discord client settings.
 type Settings struct {
 	RenderEmbeds           bool               `json:"render_embeds"`
@@ -502,6 +529,12 @@ type MessageReaction struct {
 	ChannelID string `json:"channel_id"`
 }
 
+// GatewayBotResponse stores the data for the gateway/bot response
+type GatewayBotResponse struct {
+	URL    string `json:"url"`
+	Shards int    `json:"shards"`
+}
+
 // Constants for the different bit offsets of text channel permissions
 const (
 	PermissionReadMessages = 1 << (iota + 10)
@@ -542,6 +575,8 @@ const (
 	PermissionAdministrator
 	PermissionManageChannels
 	PermissionManageServer
+	PermissionAddReactions
+	PermissionViewAuditLogs
 
 	PermissionAllText = PermissionReadMessages |
 		PermissionSendMessages |
@@ -561,9 +596,65 @@ const (
 		PermissionAllVoice |
 		PermissionCreateInstantInvite |
 		PermissionManageRoles |
-		PermissionManageChannels
+		PermissionManageChannels |
+		PermissionAddReactions |
+		PermissionViewAuditLogs
 	PermissionAll = PermissionAllChannel |
 		PermissionKickMembers |
 		PermissionBanMembers |
-		PermissionManageServer
+		PermissionManageServer |
+		PermissionAdministrator
+)
+
+// Block contains Discord JSON Error Response codes
+const (
+	ErrCodeUnknownAccount     = 10001
+	ErrCodeUnknownApplication = 10002
+	ErrCodeUnknownChannel     = 10003
+	ErrCodeUnknownGuild       = 10004
+	ErrCodeUnknownIntegration = 10005
+	ErrCodeUnknownInvite      = 10006
+	ErrCodeUnknownMember      = 10007
+	ErrCodeUnknownMessage     = 10008
+	ErrCodeUnknownOverwrite   = 10009
+	ErrCodeUnknownProvider    = 10010
+	ErrCodeUnknownRole        = 10011
+	ErrCodeUnknownToken       = 10012
+	ErrCodeUnknownUser        = 10013
+	ErrCodeUnknownEmoji       = 10014
+
+	ErrCodeBotsCannotUseEndpoint  = 20001
+	ErrCodeOnlyBotsCanUseEndpoint = 20002
+
+	ErrCodeMaximumGuildsReached     = 30001
+	ErrCodeMaximumFriendsReached    = 30002
+	ErrCodeMaximumPinsReached       = 30003
+	ErrCodeMaximumGuildRolesReached = 30005
+	ErrCodeTooManyReactions         = 30010
+
+	ErrCodeUnauthorized = 40001
+
+	ErrCodeMissingAccess                             = 50001
+	ErrCodeInvalidAccountType                        = 50002
+	ErrCodeCannotExecuteActionOnDMChannel            = 50003
+	ErrCodeEmbedCisabled                             = 50004
+	ErrCodeCannotEditFromAnotherUser                 = 50005
+	ErrCodeCannotSendEmptyMessage                    = 50006
+	ErrCodeCannotSendMessagesToThisUser              = 50007
+	ErrCodeCannotSendMessagesInVoiceChannel          = 50008
+	ErrCodeChannelVerificationLevelTooHigh           = 50009
+	ErrCodeOAuth2ApplicationDoesNotHaveBot           = 50010
+	ErrCodeOAuth2ApplicationLimitReached             = 50011
+	ErrCodeInvalidOAuthState                         = 50012
+	ErrCodeMissingPermissions                        = 50013
+	ErrCodeInvalidAuthenticationToken                = 50014
+	ErrCodeNoteTooLong                               = 50015
+	ErrCodeTooFewOrTooManyMessagesToDelete           = 50016
+	ErrCodeCanOnlyPinMessageToOriginatingChannel     = 50019
+	ErrCodeCannotExecuteActionOnSystemMessage        = 50021
+	ErrCodeMessageProvidedTooOldForBulkDelete        = 50034
+	ErrCodeInvalidFormBody                           = 50035
+	ErrCodeInviteAcceptedToGuildApplicationsBotNotIn = 50036
+
+	ErrCodeReactionBlocked = 90001
 )

vendor/github.com/bwmarrin/discordgo/tools/cmd/eventhandlers/main.go

@@ -37,18 +37,18 @@ type {{privateName .}}EventHandler func(*Session, *{{.}})
 func (eh {{privateName .}}EventHandler) Type() string {
   return {{privateName .}}EventType
 }
-
+{{if isDiscordEvent .}}
 // New returns a new instance of {{.}}.
 func (eh {{privateName .}}EventHandler) New() interface{} {
   return &{{.}}{}
-}
-
+}{{end}}
 // Handle is the handler for {{.}} events.
 func (eh {{privateName .}}EventHandler) Handle(s *Session, i interface{}) {
   if t, ok := i.(*{{.}}); ok {
     eh(s, t)
   }
 }
+
 {{end}}
 func handlerForInterface(handler interface{}) EventHandler {
   switch v := handler.(type) {
@@ -60,6 +60,7 @@ func handlerForInterface(handler interface{}) EventHandler {
 
   return nil
 }
+
 func init() { {{range .}}{{if isDiscordEvent .}}
   registerInterfaceProvider({{privateName .}}EventHandler(nil)){{end}}{{end}}
 }

vendor/github.com/bwmarrin/discordgo/user.go

@@ -0,0 +1,42 @@
+package discordgo
+
+import (
+	"fmt"
+	"strings"
+)
+
+// A User stores all data for an individual Discord user.
+type User struct {
+	ID            string `json:"id"`
+	Email         string `json:"email"`
+	Username      string `json:"username"`
+	Avatar        string `json:"avatar"`
+	Discriminator string `json:"discriminator"`
+	Token         string `json:"token"`
+	Verified      bool   `json:"verified"`
+	MFAEnabled    bool   `json:"mfa_enabled"`
+	Bot           bool   `json:"bot"`
+}
+
+// String returns a unique identifier of the form username#discriminator
+func (u *User) String() string {
+	return fmt.Sprintf("%s#%s", u.Username, u.Discriminator)
+}
+
+// Mention return a string which mentions the user
+func (u *User) Mention() string {
+	return fmt.Sprintf("<@%s>", u.ID)
+}
+
+// AvatarURL returns a URL to the user's avatar.
+//		size:     The size of the user's avatar as a power of two
+func (u *User) AvatarURL(size string) string {
+	var URL string
+	if strings.HasPrefix(u.Avatar, "a_") {
+		URL = EndpointUserAvatarAnimated(u.ID, u.Avatar)
+	} else {
+		URL = EndpointUserAvatar(u.ID, u.Avatar)
+	}
+
+	return URL + "?size=" + size
+}

vendor/github.com/bwmarrin/discordgo/voice.go

@@ -15,7 +15,6 @@ import (
 	"fmt"
 	"log"
 	"net"
-	"runtime"
 	"strings"
 	"sync"
 	"time"
@@ -93,18 +92,22 @@ func (v *VoiceConnection) Speaking(b bool) (err error) {
 	}
 
 	if v.wsConn == nil {
-		return fmt.Errorf("No VoiceConnection websocket.")
+		return fmt.Errorf("no VoiceConnection websocket")
 	}
 
 	data := voiceSpeakingOp{5, voiceSpeakingData{b, 0}}
 	v.wsMutex.Lock()
 	err = v.wsConn.WriteJSON(data)
 	v.wsMutex.Unlock()
+
+	v.Lock()
+	defer v.Unlock()
 	if err != nil {
 		v.speaking = false
 		log.Println("Speaking() write json error:", err)
 		return
 	}
+
 	v.speaking = b
 
 	return
@@ -139,9 +142,9 @@ func (v *VoiceConnection) Disconnect() (err error) {
 	// Send a OP4 with a nil channel to disconnect
 	if v.sessionID != "" {
 		data := voiceChannelJoinOp{4, voiceChannelJoinData{&v.GuildID, nil, true, true}}
-		v.wsMutex.Lock()
+		v.session.wsMutex.Lock()
 		err = v.session.wsConn.WriteJSON(data)
-		v.wsMutex.Unlock()
+		v.session.wsMutex.Unlock()
 		v.sessionID = ""
 	}
 
@@ -149,7 +152,10 @@ func (v *VoiceConnection) Disconnect() (err error) {
 	v.Close()
 
 	v.log(LogInformational, "Deleting VoiceConnection %s", v.GuildID)
+
+	v.session.Lock()
 	delete(v.session.VoiceConnections, v.GuildID)
+	v.session.Unlock()
 
 	return
 }
@@ -185,7 +191,9 @@ func (v *VoiceConnection) Close() {
 
 		// To cleanly close a connection, a client should send a close
 		// frame and wait for the server to close the connection.
+		v.wsMutex.Lock()
 		err := v.wsConn.WriteMessage(websocket.CloseMessage, websocket.FormatCloseMessage(websocket.CloseNormalClosure, ""))
+		v.wsMutex.Unlock()
 		if err != nil {
 			v.log(LogError, "error closing websocket, %s", err)
 		}
@@ -246,12 +254,15 @@ func (v *VoiceConnection) waitUntilConnected() error {
 
 	i := 0
 	for {
-		if v.Ready {
+		v.RLock()
+		ready := v.Ready
+		v.RUnlock()
+		if ready {
 			return nil
 		}
 
 		if i > 10 {
-			return fmt.Errorf("Timeout waiting for voice.")
+			return fmt.Errorf("timeout waiting for voice")
 		}
 
 		time.Sleep(1 * time.Second)
@@ -282,7 +293,7 @@ func (v *VoiceConnection) open() (err error) {
 			break
 		}
 		if i > 20 { // only loop for up to 1 second total
-			return fmt.Errorf("Did not receive voice Session ID in time.")
+			return fmt.Errorf("did not receive voice Session ID in time")
 		}
 		time.Sleep(50 * time.Millisecond)
 		i++
@@ -409,8 +420,6 @@ func (v *VoiceConnection) onEvent(message []byte) {
 			go v.opusReceiver(v.udpConn, v.close, v.OpusRecv)
 		}
 
-		// Send the ready event
-		v.connected <- true
 		return
 
 	case 3: // HEARTBEAT response
@@ -418,6 +427,9 @@ func (v *VoiceConnection) onEvent(message []byte) {
 		return
 
 	case 4: // udp encryption secret key
+		v.Lock()
+		defer v.Unlock()
+
 		v.op4 = voiceOP4{}
 		if err := json.Unmarshal(e.RawData, &v.op4); err != nil {
 			v.log(LogError, "OP4 unmarshall error, %s, %s", err, string(e.RawData))
@@ -466,6 +478,7 @@ func (v *VoiceConnection) wsHeartbeat(wsConn *websocket.Conn, close <-chan struc
 
 	var err error
 	ticker := time.NewTicker(i * time.Millisecond)
+	defer ticker.Stop()
 	for {
 		v.log(LogDebug, "sending heartbeat packet")
 		v.wsMutex.Lock()
@@ -616,6 +629,7 @@ func (v *VoiceConnection) udpKeepAlive(udpConn *net.UDPConn, close <-chan struct
 	packet := make([]byte, 8)
 
 	ticker := time.NewTicker(i)
+	defer ticker.Stop()
 	for {
 
 		binary.LittleEndian.PutUint64(packet, sequence)
@@ -644,12 +658,16 @@ func (v *VoiceConnection) opusSender(udpConn *net.UDPConn, close <-chan struct{}
 		return
 	}
 
-	runtime.LockOSThread()
-
 	// VoiceConnection is now ready to receive audio packets
 	// TODO: this needs reviewed as I think there must be a better way.
+	v.Lock()
 	v.Ready = true
-	defer func() { v.Ready = false }()
+	v.Unlock()
+	defer func() {
+		v.Lock()
+		v.Ready = false
+		v.Unlock()
+	}()
 
 	var sequence uint16
 	var timestamp uint32
@@ -665,6 +683,7 @@ func (v *VoiceConnection) opusSender(udpConn *net.UDPConn, close <-chan struct{}
 
 	// start a send loop that loops until buf chan is closed
 	ticker := time.NewTicker(time.Millisecond * time.Duration(size/(rate/1000)))
+	defer ticker.Stop()
 	for {
 
 		// Get data from chan.  If chan is closed, return.
@@ -678,7 +697,10 @@ func (v *VoiceConnection) opusSender(udpConn *net.UDPConn, close <-chan struct{}
 			// else, continue loop
 		}
 
-		if !v.speaking {
+		v.RLock()
+		speaking := v.speaking
+		v.RUnlock()
+		if !speaking {
 			err := v.Speaking(true)
 			if err != nil {
 				v.log(LogError, "error sending speaking packet, %s", err)
@@ -691,7 +713,9 @@ func (v *VoiceConnection) opusSender(udpConn *net.UDPConn, close <-chan struct{}
 
 		// encrypt the opus data
 		copy(nonce[:], udpHeader)
+		v.RLock()
 		sendbuf := secretbox.Seal(udpHeader, recvbuf, &nonce, &v.op4.SecretKey)
+		v.RUnlock()
 
 		// block here until we're exactly at the right time :)
 		// Then send rtp audio packet to Discord over UDP
@@ -742,7 +766,6 @@ func (v *VoiceConnection) opusReceiver(udpConn *net.UDPConn, close <-chan struct
 		return
 	}
 
-	p := Packet{}
 	recvbuf := make([]byte, 1024)
 	var nonce [24]byte
 
@@ -773,11 +796,12 @@ func (v *VoiceConnection) opusReceiver(udpConn *net.UDPConn, close <-chan struct
 		}
 
 		// For now, skip anything except audio.
-		if rlen < 12 || recvbuf[0] != 0x80 {
+		if rlen < 12 || (recvbuf[0] != 0x80 && recvbuf[0] != 0x90) {
 			continue
 		}
 
 		// build a audio packet struct
+		p := Packet{}
 		p.Type = recvbuf[0:2]
 		p.Sequence = binary.BigEndian.Uint16(recvbuf[2:4])
 		p.Timestamp = binary.BigEndian.Uint32(recvbuf[4:8])
@@ -786,8 +810,17 @@ func (v *VoiceConnection) opusReceiver(udpConn *net.UDPConn, close <-chan struct
 		copy(nonce[:], recvbuf[0:12])
 		p.Opus, _ = secretbox.Open(nil, recvbuf[12:rlen], &nonce, &v.op4.SecretKey)
 
+		if len(p.Opus) > 8 && recvbuf[0] == 0x90 {
+			// Extension bit is set, first 8 bytes is the extended header
+			p.Opus = p.Opus[8:]
+		}
+
 		if c != nil {
-			c <- &p
+			select {
+			case c <- &p:
+			case <-close:
+				return
+			}
 		}
 	}
 }
@@ -837,6 +870,8 @@ func (v *VoiceConnection) reconnect() {
 			return
 		}
 
+		v.log(LogInformational, "error reconnecting to channel %s, %s", v.ChannelID, err)
+
 		// if the reconnect above didn't work lets just send a disconnect
 		// packet to reset things.
 		// Send a OP4 with a nil channel to disconnect
@@ -848,6 +883,5 @@ func (v *VoiceConnection) reconnect() {
 			v.log(LogError, "error sending disconnect packet, %s", err)
 		}
 
-		v.log(LogInformational, "error reconnecting to channel %s, %s", v.ChannelID, err)
 	}
 }

vendor/github.com/bwmarrin/discordgo/wsapi.go

@@ -15,21 +15,33 @@ import (
 	"compress/zlib"
 	"encoding/json"
 	"errors"
-	"fmt"
 	"io"
 	"net/http"
 	"runtime"
+	"sync/atomic"
 	"time"
 
 	"github.com/gorilla/websocket"
 )
 
+// ErrWSAlreadyOpen is thrown when you attempt to open
+// a websocket that already is open.
+var ErrWSAlreadyOpen = errors.New("web socket already opened")
+
+// ErrWSNotFound is thrown when you attempt to use a websocket
+// that doesn't exist
+var ErrWSNotFound = errors.New("no websocket connection exists")
+
+// ErrWSShardBounds is thrown when you try to use a shard ID that is
+// less than the total shard count
+var ErrWSShardBounds = errors.New("ShardID must be less than ShardCount")
+
 type resumePacket struct {
 	Op   int `json:"op"`
 	Data struct {
 		Token     string `json:"token"`
 		SessionID string `json:"session_id"`
-		Sequence  int    `json:"seq"`
+		Sequence  int64  `json:"seq"`
 	} `json:"d"`
 }
 
@@ -57,7 +69,7 @@ func (s *Session) Open() (err error) {
 	}
 
 	if s.wsConn != nil {
-		err = errors.New("Web socket already opened.")
+		err = ErrWSAlreadyOpen
 		return
 	}
 
@@ -74,7 +86,7 @@ func (s *Session) Open() (err error) {
 		}
 
 		// Add the version and encoding to the URL
-		s.gateway = fmt.Sprintf("%s?v=4&encoding=json", s.gateway)
+		s.gateway = s.gateway + "?v=" + APIVersion + "&encoding=json"
 	}
 
 	header := http.Header{}
@@ -89,13 +101,14 @@ func (s *Session) Open() (err error) {
 		return
 	}
 
-	if s.sessionID != "" && s.sequence > 0 {
+	sequence := atomic.LoadInt64(s.sequence)
+	if s.sessionID != "" && sequence > 0 {
 
 		p := resumePacket{}
 		p.Op = 6
 		p.Data.Token = s.Token
 		p.Data.SessionID = s.sessionID
-		p.Data.Sequence = s.sequence
+		p.Data.Sequence = sequence
 
 		s.log(LogInformational, "sending resume packet to gateway")
 		err = s.wsConn.WriteJSON(p)
@@ -117,6 +130,7 @@ func (s *Session) Open() (err error) {
 	// lock.
 	s.listening = make(chan interface{})
 	go s.listen(s.wsConn, s.listening)
+	s.LastHeartbeatAck = time.Now().UTC()
 
 	s.Unlock()
 
@@ -176,14 +190,22 @@ func (s *Session) listen(wsConn *websocket.Conn, listening <-chan interface{}) {
 }
 
 type heartbeatOp struct {
-	Op   int `json:"op"`
-	Data int `json:"d"`
+	Op   int   `json:"op"`
+	Data int64 `json:"d"`
 }
 
+type helloOp struct {
+	HeartbeatInterval time.Duration `json:"heartbeat_interval"`
+	Trace             []string      `json:"_trace"`
+}
+
+// FailedHeartbeatAcks is the Number of heartbeat intervals to wait until forcing a connection restart.
+const FailedHeartbeatAcks time.Duration = 5 * time.Millisecond
+
 // heartbeat sends regular heartbeats to Discord so it knows the client
 // is still connected.  If you do not send these heartbeats Discord will
 // disconnect the websocket connection after a few seconds.
-func (s *Session) heartbeat(wsConn *websocket.Conn, listening <-chan interface{}, i time.Duration) {
+func (s *Session) heartbeat(wsConn *websocket.Conn, listening <-chan interface{}, heartbeatIntervalMsec time.Duration) {
 
 	s.log(LogInformational, "called")
 
@@ -192,19 +214,26 @@ func (s *Session) heartbeat(wsConn *websocket.Conn, listening <-chan interface{}
 	}
 
 	var err error
-	ticker := time.NewTicker(i * time.Millisecond)
+	ticker := time.NewTicker(heartbeatIntervalMsec * time.Millisecond)
+	defer ticker.Stop()
 
 	for {
-
-		s.log(LogInformational, "sending gateway websocket heartbeat seq %d", s.sequence)
+		s.RLock()
+		last := s.LastHeartbeatAck
+		s.RUnlock()
+		sequence := atomic.LoadInt64(s.sequence)
+		s.log(LogInformational, "sending gateway websocket heartbeat seq %d", sequence)
 		s.wsMutex.Lock()
-		err = wsConn.WriteJSON(heartbeatOp{1, s.sequence})
+		err = wsConn.WriteJSON(heartbeatOp{1, sequence})
 		s.wsMutex.Unlock()
-		if err != nil {
-			s.log(LogError, "error sending heartbeat to gateway %s, %s", s.gateway, err)
-			s.Lock()
-			s.DataReady = false
-			s.Unlock()
+		if err != nil || time.Now().UTC().Sub(last) > (heartbeatIntervalMsec*FailedHeartbeatAcks) {
+			if err != nil {
+				s.log(LogError, "error sending heartbeat to gateway %s, %s", s.gateway, err)
+			} else {
+				s.log(LogError, "haven't gotten a heartbeat ACK in %v, triggering a reconnection", time.Now().UTC().Sub(last))
+			}
+			s.Close()
+			s.reconnect()
 			return
 		}
 		s.Lock()
@@ -221,8 +250,10 @@ func (s *Session) heartbeat(wsConn *websocket.Conn, listening <-chan interface{}
 }
 
 type updateStatusData struct {
-	IdleSince *int  `json:"idle_since"`
-	Game      *Game `json:"game"`
+	IdleSince *int   `json:"since"`
+	Game      *Game  `json:"game"`
+	AFK       bool   `json:"afk"`
+	Status    string `json:"status"`
 }
 
 type updateStatusOp struct {
@@ -242,10 +273,13 @@ func (s *Session) UpdateStreamingStatus(idle int, game string, url string) (err
 	s.RLock()
 	defer s.RUnlock()
 	if s.wsConn == nil {
-		return errors.New("no websocket connection exists")
+		return ErrWSNotFound
+	}
+
+	usd := updateStatusData{
+		Status: "online",
 	}
 
-	var usd updateStatusData
 	if idle > 0 {
 		usd.IdleSince = &idle
 	}
@@ -299,7 +333,7 @@ func (s *Session) RequestGuildMembers(guildID, query string, limit int) (err err
 	s.RLock()
 	defer s.RUnlock()
 	if s.wsConn == nil {
-		return errors.New("no websocket connection exists")
+		return ErrWSNotFound
 	}
 
 	data := requestGuildMembersData{
@@ -365,7 +399,7 @@ func (s *Session) onEvent(messageType int, message []byte) {
 	if e.Operation == 1 {
 		s.log(LogInformational, "sending heartbeat in response to Op1")
 		s.wsMutex.Lock()
-		err = s.wsConn.WriteJSON(heartbeatOp{1, s.sequence})
+		err = s.wsConn.WriteJSON(heartbeatOp{1, atomic.LoadInt64(s.sequence)})
 		s.wsMutex.Unlock()
 		if err != nil {
 			s.log(LogError, "error sending heartbeat in response to Op1")
@@ -378,7 +412,10 @@ func (s *Session) onEvent(messageType int, message []byte) {
 	// Reconnect
 	// Must immediately disconnect from gateway and reconnect to new gateway.
 	if e.Operation == 7 {
-		// TODO
+		s.log(LogInformational, "Closing and reconnecting in response to Op7")
+		s.Close()
+		s.reconnect()
+		return
 	}
 
 	// Invalid Session
@@ -396,6 +433,24 @@ func (s *Session) onEvent(messageType int, message []byte) {
 		return
 	}
 
+	if e.Operation == 10 {
+		var h helloOp
+		if err = json.Unmarshal(e.RawData, &h); err != nil {
+			s.log(LogError, "error unmarshalling helloOp, %s", err)
+		} else {
+			go s.heartbeat(s.wsConn, s.listening, h.HeartbeatInterval)
+		}
+		return
+	}
+
+	if e.Operation == 11 {
+		s.Lock()
+		s.LastHeartbeatAck = time.Now().UTC()
+		s.Unlock()
+		s.log(LogInformational, "got heartbeat ACK")
+		return
+	}
+
 	// Do not try to Dispatch a non-Dispatch Message
 	if e.Operation != 0 {
 		// But we probably should be doing something with them.
@@ -405,7 +460,7 @@ func (s *Session) onEvent(messageType int, message []byte) {
 	}
 
 	// Store the message sequence
-	s.sequence = e.Sequence
+	atomic.StoreInt64(s.sequence, e.Sequence)
 
 	// Map event to registered event handlers and pass it along to any registered handlers.
 	if eh, ok := registeredInterfaceProviders[e.Type]; ok {
@@ -458,18 +513,24 @@ func (s *Session) ChannelVoiceJoin(gID, cID string, mute, deaf bool) (voice *Voi
 
 	s.log(LogInformational, "called")
 
+	s.RLock()
 	voice, _ = s.VoiceConnections[gID]
+	s.RUnlock()
 
 	if voice == nil {
 		voice = &VoiceConnection{}
+		s.Lock()
 		s.VoiceConnections[gID] = voice
+		s.Unlock()
 	}
 
+	voice.Lock()
 	voice.GuildID = gID
 	voice.ChannelID = cID
 	voice.deaf = deaf
 	voice.mute = mute
 	voice.session = s
+	voice.Unlock()
 
 	// Send the request to Discord that we want to join the voice channel
 	data := voiceChannelJoinOp{4, voiceChannelJoinData{&gID, &cID, mute, deaf}}
@@ -500,7 +561,9 @@ func (s *Session) onVoiceStateUpdate(st *VoiceStateUpdate) {
 	}
 
 	// Check if we have a voice connection to update
+	s.RLock()
 	voice, exists := s.VoiceConnections[st.GuildID]
+	s.RUnlock()
 	if !exists {
 		return
 	}
@@ -511,8 +574,11 @@ func (s *Session) onVoiceStateUpdate(st *VoiceStateUpdate) {
 	}
 
 	// Store the SessionID for later use.
+	voice.Lock()
 	voice.UserID = st.UserID
 	voice.sessionID = st.SessionID
+	voice.ChannelID = st.ChannelID
+	voice.Unlock()
 }
 
 // onVoiceServerUpdate handles the Voice Server Update data websocket event.
@@ -524,7 +590,9 @@ func (s *Session) onVoiceServerUpdate(st *VoiceServerUpdate) {
 
 	s.log(LogInformational, "called")
 
+	s.RLock()
 	voice, exists := s.VoiceConnections[st.GuildID]
+	s.RUnlock()
 
 	// If no VoiceConnection exists, just skip this
 	if !exists {
@@ -536,9 +604,11 @@ func (s *Session) onVoiceServerUpdate(st *VoiceServerUpdate) {
 	voice.Close()
 
 	// Store values for later use
+	voice.Lock()
 	voice.token = st.Token
 	voice.endpoint = st.Endpoint
 	voice.GuildID = st.GuildID
+	voice.Unlock()
 
 	// Open a conenction to the voice server
 	err := voice.open()
@@ -588,7 +658,7 @@ func (s *Session) identify() error {
 	if s.ShardCount > 1 {
 
 		if s.ShardID >= s.ShardCount {
-			return errors.New("ShardID must be less than ShardCount")
+			return ErrWSShardBounds
 		}
 
 		data.Shard = &[2]int{s.ShardID, s.ShardCount}
@@ -628,6 +698,8 @@ func (s *Session) reconnect() {
 				// However, there seems to be cases where something "weird"
 				// happens.  So we're doing this for now just to improve
 				// stability in those edge cases.
+				s.RLock()
+				defer s.RUnlock()
 				for _, v := range s.VoiceConnections {
 
 					s.log(LogInformational, "reconnecting voice connection to guild %s", v.GuildID)
@@ -641,6 +713,13 @@ func (s *Session) reconnect() {
 				return
 			}
 
+			// Certain race conditions can call reconnect() twice. If this happens, we
+			// just break out of the reconnect loop
+			if err == ErrWSAlreadyOpen {
+				s.log(LogInformational, "Websocket already exists, no need to reconnect")
+				return
+			}
+
 			s.log(LogError, "error reconnecting to gateway, %s", err)
 
 			<-time.After(wait * time.Second)
@@ -675,7 +754,9 @@ func (s *Session) Close() (err error) {
 		s.log(LogInformational, "sending close frame")
 		// To cleanly close a connection, a client should send a close
 		// frame and wait for the server to close the connection.
+		s.wsMutex.Lock()
 		err := s.wsConn.WriteMessage(websocket.CloseMessage, websocket.FormatCloseMessage(websocket.CloseNormalClosure, ""))
+		s.wsMutex.Unlock()
 		if err != nil {
 			s.log(LogInformational, "error closing websocket, %s", err)
 		}

vendor/github.com/go-telegram-bot-api/telegram-bot-api/bot.go

@@ -191,7 +191,11 @@ func (bot *BotAPI) UploadFile(endpoint string, params map[string]string, fieldna
 	}
 
 	var apiResp APIResponse
-	json.Unmarshal(bytes, &apiResp)
+
+	err = json.Unmarshal(bytes, &apiResp)
+	if err != nil {
+		return APIResponse{}, err
+	}
 
 	if !apiResp.Ok {
 		return APIResponse{}, errors.New(apiResp.Description)
@@ -438,14 +442,7 @@ func (bot *BotAPI) SetWebhook(config WebhookConfig) (APIResponse, error) {
 		return APIResponse{}, err
 	}
 
-	var apiResp APIResponse
-	json.Unmarshal(resp.Result, &apiResp)
-
-	if bot.Debug {
-		log.Printf("setWebhook resp: %+v\n", apiResp)
-	}
-
-	return apiResp, nil
+	return resp, nil
 }
 
 // GetWebhookInfo allows you to fetch information about a webhook and if
@@ -550,7 +547,7 @@ func (bot *BotAPI) AnswerCallbackQuery(config CallbackConfig) (APIResponse, erro
 // KickChatMember kicks a user from a chat. Note that this only will work
 // in supergroups, and requires the bot to be an admin. Also note they
 // will be unable to rejoin until they are unbanned.
-func (bot *BotAPI) KickChatMember(config ChatMemberConfig) (APIResponse, error) {
+func (bot *BotAPI) KickChatMember(config KickChatMemberConfig) (APIResponse, error) {
 	v := url.Values{}
 
 	if config.SuperGroupUsername == "" {
@@ -560,6 +557,10 @@ func (bot *BotAPI) KickChatMember(config ChatMemberConfig) (APIResponse, error)
 	}
 	v.Add("user_id", strconv.Itoa(config.UserID))
 
+	if config.UntilDate != 0 {
+		v.Add("until_date", strconv.FormatInt(config.UntilDate, 10))
+	}
+
 	bot.debugLog("kickChatMember", v, nil)
 
 	return bot.MakeRequest("kickChatMember", v)
@@ -677,14 +678,16 @@ func (bot *BotAPI) GetChatMember(config ChatConfigWithUser) (ChatMember, error)
 }
 
 // UnbanChatMember unbans a user from a chat. Note that this only will work
-// in supergroups, and requires the bot to be an admin.
+// in supergroups and channels, and requires the bot to be an admin.
 func (bot *BotAPI) UnbanChatMember(config ChatMemberConfig) (APIResponse, error) {
 	v := url.Values{}
 
-	if config.SuperGroupUsername == "" {
-		v.Add("chat_id", strconv.FormatInt(config.ChatID, 10))
-	} else {
+	if config.SuperGroupUsername != "" {
 		v.Add("chat_id", config.SuperGroupUsername)
+	} else if config.ChannelUsername != "" {
+		v.Add("chat_id", config.ChannelUsername)
+	} else {
+		v.Add("chat_id", strconv.FormatInt(config.ChatID, 10))
 	}
 	v.Add("user_id", strconv.Itoa(config.UserID))
 
@@ -693,6 +696,82 @@ func (bot *BotAPI) UnbanChatMember(config ChatMemberConfig) (APIResponse, error)
 	return bot.MakeRequest("unbanChatMember", v)
 }
 
+// RestrictChatMember to restrict a user in a supergroup. The bot must be an
+//administrator in the supergroup for this to work and must have the
+//appropriate admin rights. Pass True for all boolean parameters to lift
+//restrictions from a user. Returns True on success.
+func (bot *BotAPI) RestrictChatMember(config RestrictChatMemberConfig) (APIResponse, error) {
+	v := url.Values{}
+
+	if config.SuperGroupUsername != "" {
+		v.Add("chat_id", config.SuperGroupUsername)
+	} else if config.ChannelUsername != "" {
+		v.Add("chat_id", config.ChannelUsername)
+	} else {
+		v.Add("chat_id", strconv.FormatInt(config.ChatID, 10))
+	}
+	v.Add("user_id", strconv.Itoa(config.UserID))
+
+	if &config.CanSendMessages != nil {
+		v.Add("can_send_messages", strconv.FormatBool(*config.CanSendMessages))
+	}
+	if &config.CanSendMediaMessages != nil {
+		v.Add("can_send_media_messages", strconv.FormatBool(*config.CanSendMediaMessages))
+	}
+	if &config.CanSendOtherMessages != nil {
+		v.Add("can_send_other_messages", strconv.FormatBool(*config.CanSendOtherMessages))
+	}
+	if &config.CanAddWebPagePreviews != nil {
+		v.Add("can_add_web_page_previews", strconv.FormatBool(*config.CanAddWebPagePreviews))
+	}
+
+	bot.debugLog("restrictChatMember", v, nil)
+
+	return bot.MakeRequest("restrictChatMember", v)
+}
+
+func (bot *BotAPI) PromoteChatMember(config PromoteChatMemberConfig) (APIResponse, error) {
+	v := url.Values{}
+
+	if config.SuperGroupUsername != "" {
+		v.Add("chat_id", config.SuperGroupUsername)
+	} else if config.ChannelUsername != "" {
+		v.Add("chat_id", config.ChannelUsername)
+	} else {
+		v.Add("chat_id", strconv.FormatInt(config.ChatID, 10))
+	}
+	v.Add("user_id", strconv.Itoa(config.UserID))
+
+	if &config.CanChangeInfo != nil {
+		v.Add("can_change_info", strconv.FormatBool(*config.CanChangeInfo))
+	}
+	if &config.CanPostMessages != nil {
+		v.Add("can_post_messages", strconv.FormatBool(*config.CanPostMessages))
+	}
+	if &config.CanEditMessages != nil {
+		v.Add("can_edit_messages", strconv.FormatBool(*config.CanEditMessages))
+	}
+	if &config.CanDeleteMessages != nil {
+		v.Add("can_delete_messages", strconv.FormatBool(*config.CanDeleteMessages))
+	}
+	if &config.CanInviteUsers != nil {
+		v.Add("can_invite_users", strconv.FormatBool(*config.CanInviteUsers))
+	}
+	if &config.CanRestrictMembers != nil {
+		v.Add("can_restrict_members", strconv.FormatBool(*config.CanRestrictMembers))
+	}
+	if &config.CanPinMessages != nil {
+		v.Add("can_pin_messages", strconv.FormatBool(*config.CanPinMessages))
+	}
+	if &config.CanPromoteMembers != nil {
+		v.Add("can_promote_members", strconv.FormatBool(*config.CanPromoteMembers))
+	}
+
+	bot.debugLog("promoteChatMember", v, nil)
+
+	return bot.MakeRequest("promoteChatMember", v)
+}
+
 // GetGameHighScores allows you to get the high scores for a game.
 func (bot *BotAPI) GetGameHighScores(config GetGameHighScoresConfig) ([]GameHighScore, error) {
 	v, _ := config.values()
@@ -707,3 +786,93 @@ func (bot *BotAPI) GetGameHighScores(config GetGameHighScoresConfig) ([]GameHigh
 
 	return highScores, err
 }
+
+// AnswerShippingQuery allows you to reply to Update with shipping_query parameter.
+func (bot *BotAPI) AnswerShippingQuery(config ShippingConfig) (APIResponse, error) {
+	v := url.Values{}
+
+	v.Add("shipping_query_id", config.ShippingQueryID)
+	v.Add("ok", strconv.FormatBool(config.OK))
+	if config.OK == true {
+		data, err := json.Marshal(config.ShippingOptions)
+		if err != nil {
+			return APIResponse{}, err
+		}
+		v.Add("shipping_options", string(data))
+	} else {
+		v.Add("error_message", config.ErrorMessage)
+	}
+
+	bot.debugLog("answerShippingQuery", v, nil)
+
+	return bot.MakeRequest("answerShippingQuery", v)
+}
+
+// AnswerPreCheckoutQuery allows you to reply to Update with pre_checkout_query.
+func (bot *BotAPI) AnswerPreCheckoutQuery(config PreCheckoutConfig) (APIResponse, error) {
+	v := url.Values{}
+
+	v.Add("pre_checkout_query_id", config.PreCheckoutQueryID)
+	v.Add("ok", strconv.FormatBool(config.OK))
+	if config.OK != true {
+		v.Add("error", config.ErrorMessage)
+	}
+
+	bot.debugLog("answerPreCheckoutQuery", v, nil)
+
+	return bot.MakeRequest("answerPreCheckoutQuery", v)
+}
+
+// DeleteMessage deletes a message in a chat
+func (bot *BotAPI) DeleteMessage(config DeleteMessageConfig) (APIResponse, error) {
+	v, err := config.values()
+	if err != nil {
+		return APIResponse{}, err
+	}
+
+	bot.debugLog(config.method(), v, nil)
+
+	return bot.MakeRequest(config.method(), v)
+}
+
+// GetInviteLink get InviteLink for a chat
+func (bot *BotAPI) GetInviteLink(config ChatConfig) (string, error) {
+	v := url.Values{}
+
+	if config.SuperGroupUsername == "" {
+		v.Add("chat_id", strconv.FormatInt(config.ChatID, 10))
+	} else {
+		v.Add("chat_id", config.SuperGroupUsername)
+	}
+
+	resp, err := bot.MakeRequest("exportChatInviteLink", v)
+
+	var inviteLink string
+	err = json.Unmarshal(resp.Result, &inviteLink)
+
+	return inviteLink, err
+}
+
+// Pin message in supergroup
+func (bot *BotAPI) PinChatMessage(config PinChatMessageConfig) (APIResponse, error) {
+	v, err := config.values()
+	if err != nil {
+		return APIResponse{}, err
+	}
+
+	bot.debugLog(config.method(), v, nil)
+
+	return bot.MakeRequest(config.method(), v)
+}
+
+// Unpin message in supergroup
+func (bot *BotAPI) UnpinChatMessage(config UnpinChatMessageConfig) (APIResponse, error) {
+	v, err := config.values()
+	if err != nil {
+		return APIResponse{}, err
+	}
+
+	bot.debugLog(config.method(), v, nil)
+
+	return bot.MakeRequest(config.method(), v)
+}

vendor/github.com/go-telegram-bot-api/telegram-bot-api/configs.go

@@ -349,6 +349,7 @@ func (config AudioConfig) method() string {
 // DocumentConfig contains information about a SendDocument request.
 type DocumentConfig struct {
 	BaseFile
+	Caption string
 }
 
 // values returns a url.Values representation of DocumentConfig.
@@ -359,6 +360,9 @@ func (config DocumentConfig) values() (url.Values, error) {
 	}
 
 	v.Add(config.name(), config.FileID)
+	if config.Caption != "" {
+		v.Add("caption", config.Caption)
+	}
 
 	return v, nil
 }
@@ -367,6 +371,10 @@ func (config DocumentConfig) values() (url.Values, error) {
 func (config DocumentConfig) params() (map[string]string, error) {
 	params, _ := config.BaseFile.params()
 
+	if config.Caption != "" {
+		params["caption"] = config.Caption
+	}
+
 	return params, nil
 }
 
@@ -443,6 +451,10 @@ func (config VideoConfig) values() (url.Values, error) {
 func (config VideoConfig) params() (map[string]string, error) {
 	params, _ := config.BaseFile.params()
 
+	if config.Caption != "" {
+		params["caption"] = config.Caption
+	}
+
 	return params, nil
 }
 
@@ -456,6 +468,57 @@ func (config VideoConfig) method() string {
 	return "sendVideo"
 }
 
+// VideoNoteConfig contains information about a SendVideoNote request.
+type VideoNoteConfig struct {
+	BaseFile
+	Duration int
+	Length   int
+}
+
+// values returns a url.Values representation of VideoNoteConfig.
+func (config VideoNoteConfig) values() (url.Values, error) {
+	v, err := config.BaseChat.values()
+	if err != nil {
+		return v, err
+	}
+
+	v.Add(config.name(), config.FileID)
+	if config.Duration != 0 {
+		v.Add("duration", strconv.Itoa(config.Duration))
+	}
+
+	// Telegram API seems to have a bug, if no length is provided or it is 0, it will send an error response
+	if config.Length != 0 {
+		v.Add("length", strconv.Itoa(config.Length))
+	}
+
+	return v, nil
+}
+
+// params returns a map[string]string representation of VideoNoteConfig.
+func (config VideoNoteConfig) params() (map[string]string, error) {
+	params, _ := config.BaseFile.params()
+
+	if config.Length != 0 {
+		params["length"] = strconv.Itoa(config.Length)
+	}
+	if config.Duration != 0 {
+		params["duration"] = strconv.Itoa(config.Duration)
+	}
+
+	return params, nil
+}
+
+// name returns the field name for the VideoNote.
+func (config VideoNoteConfig) name() string {
+	return "video_note"
+}
+
+// method returns Telegram API method name for sending VideoNote.
+func (config VideoNoteConfig) method() string {
+	return "sendVideoNote"
+}
+
 // VoiceConfig contains information about a SendVoice request.
 type VoiceConfig struct {
 	BaseFile
@@ -474,6 +537,9 @@ func (config VoiceConfig) values() (url.Values, error) {
 	if config.Duration != 0 {
 		v.Add("duration", strconv.Itoa(config.Duration))
 	}
+	if config.Caption != "" {
+		v.Add("caption", config.Caption)
+	}
 
 	return v, nil
 }
@@ -485,6 +551,9 @@ func (config VoiceConfig) params() (map[string]string, error) {
 	if config.Duration != 0 {
 		params["duration"] = strconv.Itoa(config.Duration)
 	}
+	if config.Caption != "" {
+		params["caption"] = config.Caption
+	}
 
 	return params, nil
 }
@@ -814,9 +883,39 @@ type CallbackConfig struct {
 type ChatMemberConfig struct {
 	ChatID             int64
 	SuperGroupUsername string
+	ChannelUsername    string
 	UserID             int
 }
 
+// KickChatMemberConfig contains extra fields to kick user
+type KickChatMemberConfig struct {
+	ChatMemberConfig
+	UntilDate int64
+}
+
+// RestrictChatMemberConfig contains fields to restrict members of chat
+type RestrictChatMemberConfig struct {
+	ChatMemberConfig
+	UntilDate             int64
+	CanSendMessages       *bool
+	CanSendMediaMessages  *bool
+	CanSendOtherMessages  *bool
+	CanAddWebPagePreviews *bool
+}
+
+// PromoteChatMemberConfig contains fields to promote members of chat
+type PromoteChatMemberConfig struct {
+	ChatMemberConfig
+	CanChangeInfo      *bool
+	CanPostMessages    *bool
+	CanEditMessages    *bool
+	CanDeleteMessages  *bool
+	CanInviteUsers     *bool
+	CanRestrictMembers *bool
+	CanPinMessages     *bool
+	CanPromoteMembers  *bool
+}
+
 // ChatConfig contains information about getting information on a chat.
 type ChatConfig struct {
 	ChatID             int64
@@ -830,3 +929,147 @@ type ChatConfigWithUser struct {
 	SuperGroupUsername string
 	UserID             int
 }
+
+// InvoiceConfig contains information for sendInvoice request.
+type InvoiceConfig struct {
+	BaseChat
+	Title               string          // required
+	Description         string          // required
+	Payload             string          // required
+	ProviderToken       string          // required
+	StartParameter      string          // required
+	Currency            string          // required
+	Prices              *[]LabeledPrice // required
+	PhotoURL            string
+	PhotoSize           int
+	PhotoWidth          int
+	PhotoHeight         int
+	NeedName            bool
+	NeedPhoneNumber     bool
+	NeedEmail           bool
+	NeedShippingAddress bool
+	IsFlexible          bool
+}
+
+func (config InvoiceConfig) values() (url.Values, error) {
+	v, err := config.BaseChat.values()
+	if err != nil {
+		return v, err
+	}
+	v.Add("title", config.Title)
+	v.Add("description", config.Description)
+	v.Add("payload", config.Payload)
+	v.Add("provider_token", config.ProviderToken)
+	v.Add("start_parameter", config.StartParameter)
+	v.Add("currency", config.Currency)
+	data, err := json.Marshal(config.Prices)
+	if err != nil {
+		return v, err
+	}
+	v.Add("prices", string(data))
+	if config.PhotoURL != "" {
+		v.Add("photo_url", config.PhotoURL)
+	}
+	if config.PhotoSize != 0 {
+		v.Add("photo_size", strconv.Itoa(config.PhotoSize))
+	}
+	if config.PhotoWidth != 0 {
+		v.Add("photo_width", strconv.Itoa(config.PhotoWidth))
+	}
+	if config.PhotoHeight != 0 {
+		v.Add("photo_height", strconv.Itoa(config.PhotoHeight))
+	}
+	if config.NeedName != false {
+		v.Add("need_name", strconv.FormatBool(config.NeedName))
+	}
+	if config.NeedPhoneNumber != false {
+		v.Add("need_phone_number", strconv.FormatBool(config.NeedPhoneNumber))
+	}
+	if config.NeedEmail != false {
+		v.Add("need_email", strconv.FormatBool(config.NeedEmail))
+	}
+	if config.NeedShippingAddress != false {
+		v.Add("need_shipping_address", strconv.FormatBool(config.NeedShippingAddress))
+	}
+	if config.IsFlexible != false {
+		v.Add("is_flexible", strconv.FormatBool(config.IsFlexible))
+	}
+
+	return v, nil
+}
+
+func (config InvoiceConfig) method() string {
+	return "sendInvoice"
+}
+
+// ShippingConfig contains information for answerShippingQuery request.
+type ShippingConfig struct {
+	ShippingQueryID string // required
+	OK              bool   // required
+	ShippingOptions *[]ShippingOption
+	ErrorMessage    string
+}
+
+// PreCheckoutConfig conatins information for answerPreCheckoutQuery request.
+type PreCheckoutConfig struct {
+	PreCheckoutQueryID string // required
+	OK                 bool   // required
+	ErrorMessage       string
+}
+
+// DeleteMessageConfig contains information of a message in a chat to delete.
+type DeleteMessageConfig struct {
+	ChatID    int64
+	MessageID int
+}
+
+func (config DeleteMessageConfig) method() string {
+	return "deleteMessage"
+}
+
+func (config DeleteMessageConfig) values() (url.Values, error) {
+	v := url.Values{}
+
+	v.Add("chat_id", strconv.FormatInt(config.ChatID, 10))
+	v.Add("message_id", strconv.Itoa(config.MessageID))
+
+	return v, nil
+}
+
+// PinChatMessageConfig contains information of a message in a chat to pin.
+type PinChatMessageConfig struct {
+	ChatID int64
+	MessageID int
+	DisableNotification bool
+}
+
+func (config PinChatMessageConfig) method() string {
+	return "pinChatMessage"
+}
+
+func (config PinChatMessageConfig) values() (url.Values, error) {
+	v := url.Values{}
+
+	v.Add("chat_id", strconv.FormatInt(config.ChatID, 10))
+	v.Add("message_id", strconv.Itoa(config.MessageID))
+	v.Add("disable_notification", strconv.FormatBool(config.DisableNotification))
+
+	return v, nil
+}
+
+// UnpinChatMessageConfig contains information of chat to unpin.
+type UnpinChatMessageConfig struct {
+	ChatID int64
+}
+
+func (config UnpinChatMessageConfig) method() string {
+	return "unpinChatMessage"
+}
+
+func (config UnpinChatMessageConfig) values() (url.Values, error) {
+	v := url.Values{}
+
+	v.Add("chat_id", strconv.FormatInt(config.ChatID, 10))
+
+	return v, nil
+}

vendor/github.com/go-telegram-bot-api/telegram-bot-api/helpers.go

@@ -194,6 +194,37 @@ func NewVideoShare(chatID int64, fileID string) VideoConfig {
 	}
 }
 
+// NewVideoNoteUpload creates a new video note uploader.
+//
+// chatID is where to send it, file is a string path to the file,
+// FileReader, or FileBytes.
+func NewVideoNoteUpload(chatID int64, length int, file interface{}) VideoNoteConfig {
+	return VideoNoteConfig{
+		BaseFile: BaseFile{
+			BaseChat:    BaseChat{ChatID: chatID},
+			File:        file,
+			UseExisting: false,
+		},
+		Length: length,
+	}
+}
+
+// NewVideoNoteShare shares an existing video.
+// You may use this to reshare an existing video without reuploading it.
+//
+// chatID is where to send it, fileID is the ID of the video
+// already uploaded.
+func NewVideoNoteShare(chatID int64, length int, fileID string) VideoNoteConfig {
+	return VideoNoteConfig{
+		BaseFile: BaseFile{
+			BaseChat:    BaseChat{ChatID: chatID},
+			FileID:      fileID,
+			UseExisting: true,
+		},
+		Length: length,
+	}
+}
+
 // NewVoiceUpload creates a new voice uploader.
 //
 // chatID is where to send it, file is a string path to the file,
@@ -609,3 +640,16 @@ func NewCallbackWithAlert(id, text string) CallbackConfig {
 		ShowAlert:       true,
 	}
 }
+
+// NewInvoice created a new Invoice request to the user.
+func NewInvoice(chatID int64, title, description, payload, providerToken, startParameter, currency string, prices *[]LabeledPrice) InvoiceConfig {
+	return InvoiceConfig{
+		BaseChat:       BaseChat{ChatID: chatID},
+		Title:          title,
+		Description:    description,
+		Payload:        payload,
+		ProviderToken:  providerToken,
+		StartParameter: startParameter,
+		Currency:       currency,
+		Prices:         prices}
+}

vendor/github.com/go-telegram-bot-api/telegram-bot-api/types.go

@@ -35,6 +35,8 @@ type Update struct {
 	InlineQuery        *InlineQuery        `json:"inline_query"`
 	ChosenInlineResult *ChosenInlineResult `json:"chosen_inline_result"`
 	CallbackQuery      *CallbackQuery      `json:"callback_query"`
+	ShippingQuery      *ShippingQuery      `json:"shipping_query"`
+	PreCheckoutQuery   *PreCheckoutQuery   `json:"pre_checkout_query"`
 }
 
 // UpdatesChannel is the channel for getting updates.
@@ -49,10 +51,11 @@ func (ch UpdatesChannel) Clear() {
 
 // User is a user on Telegram.
 type User struct {
-	ID        int    `json:"id"`
-	FirstName string `json:"first_name"`
-	LastName  string `json:"last_name"` // optional
-	UserName  string `json:"username"`  // optional
+	ID           int    `json:"id"`
+	FirstName    string `json:"first_name"`
+	LastName     string `json:"last_name"`     // optional
+	UserName     string `json:"username"`      // optional
+	LanguageCode string `json:"language_code"` // optional
 }
 
 // String displays a simple text version of a user.
@@ -78,15 +81,24 @@ type GroupChat struct {
 	Title string `json:"title"`
 }
 
+// ChatPhoto represents a chat photo.
+type ChatPhoto struct {
+	SmallFileID string `json:"small_file_id"`
+	BigFileID   string `json:"big_file_id"`
+}
+
 // Chat contains information about the place a message was sent.
 type Chat struct {
-	ID                  int64  `json:"id"`
-	Type                string `json:"type"`
-	Title               string `json:"title"`                          // optional
-	UserName            string `json:"username"`                       // optional
-	FirstName           string `json:"first_name"`                     // optional
-	LastName            string `json:"last_name"`                      // optional
-	AllMembersAreAdmins bool   `json:"all_members_are_administrators"` // optional
+	ID                  int64      `json:"id"`
+	Type                string     `json:"type"`
+	Title               string     `json:"title"`                          // optional
+	UserName            string     `json:"username"`                       // optional
+	FirstName           string     `json:"first_name"`                     // optional
+	LastName            string     `json:"last_name"`                      // optional
+	AllMembersAreAdmins bool       `json:"all_members_are_administrators"` // optional
+	Photo               *ChatPhoto `json:"photo"`
+	Description         string     `json:"description,omitempty"` // optional
+	InviteLink          string     `json:"invite_link,omitempty"` // optional
 }
 
 // IsPrivate returns if the Chat is a private conversation.
@@ -117,40 +129,43 @@ func (c Chat) ChatConfig() ChatConfig {
 // Message is returned by almost every request, and contains data about
 // almost anything.
 type Message struct {
-	MessageID             int              `json:"message_id"`
-	From                  *User            `json:"from"` // optional
-	Date                  int              `json:"date"`
-	Chat                  *Chat            `json:"chat"`
-	ForwardFrom           *User            `json:"forward_from"`            // optional
-	ForwardFromChat       *Chat            `json:"forward_from_chat"`       // optional
-	ForwardFromMessageID  int              `json:"forward_from_message_id"` // optional
-	ForwardDate           int              `json:"forward_date"`            // optional
-	ReplyToMessage        *Message         `json:"reply_to_message"`        // optional
-	EditDate              int              `json:"edit_date"`               // optional
-	Text                  string           `json:"text"`                    // optional
-	Entities              *[]MessageEntity `json:"entities"`                // optional
-	Audio                 *Audio           `json:"audio"`                   // optional
-	Document              *Document        `json:"document"`                // optional
-	Game                  *Game            `json:"game"`                    // optional
-	Photo                 *[]PhotoSize     `json:"photo"`                   // optional
-	Sticker               *Sticker         `json:"sticker"`                 // optional
-	Video                 *Video           `json:"video"`                   // optional
-	Voice                 *Voice           `json:"voice"`                   // optional
-	Caption               string           `json:"caption"`                 // optional
-	Contact               *Contact         `json:"contact"`                 // optional
-	Location              *Location        `json:"location"`                // optional
-	Venue                 *Venue           `json:"venue"`                   // optional
-	NewChatMember         *User            `json:"new_chat_member"`         // optional
-	LeftChatMember        *User            `json:"left_chat_member"`        // optional
-	NewChatTitle          string           `json:"new_chat_title"`          // optional
-	NewChatPhoto          *[]PhotoSize     `json:"new_chat_photo"`          // optional
-	DeleteChatPhoto       bool             `json:"delete_chat_photo"`       // optional
-	GroupChatCreated      bool             `json:"group_chat_created"`      // optional
-	SuperGroupChatCreated bool             `json:"supergroup_chat_created"` // optional
-	ChannelChatCreated    bool             `json:"channel_chat_created"`    // optional
-	MigrateToChatID       int64            `json:"migrate_to_chat_id"`      // optional
-	MigrateFromChatID     int64            `json:"migrate_from_chat_id"`    // optional
-	PinnedMessage         *Message         `json:"pinned_message"`          // optional
+	MessageID             int                `json:"message_id"`
+	From                  *User              `json:"from"` // optional
+	Date                  int                `json:"date"`
+	Chat                  *Chat              `json:"chat"`
+	ForwardFrom           *User              `json:"forward_from"`            // optional
+	ForwardFromChat       *Chat              `json:"forward_from_chat"`       // optional
+	ForwardFromMessageID  int                `json:"forward_from_message_id"` // optional
+	ForwardDate           int                `json:"forward_date"`            // optional
+	ReplyToMessage        *Message           `json:"reply_to_message"`        // optional
+	EditDate              int                `json:"edit_date"`               // optional
+	Text                  string             `json:"text"`                    // optional
+	Entities              *[]MessageEntity   `json:"entities"`                // optional
+	Audio                 *Audio             `json:"audio"`                   // optional
+	Document              *Document          `json:"document"`                // optional
+	Game                  *Game              `json:"game"`                    // optional
+	Photo                 *[]PhotoSize       `json:"photo"`                   // optional
+	Sticker               *Sticker           `json:"sticker"`                 // optional
+	Video                 *Video             `json:"video"`                   // optional
+	VideoNote             *VideoNote         `json:"video_note"`              // optional
+	Voice                 *Voice             `json:"voice"`                   // optional
+	Caption               string             `json:"caption"`                 // optional
+	Contact               *Contact           `json:"contact"`                 // optional
+	Location              *Location          `json:"location"`                // optional
+	Venue                 *Venue             `json:"venue"`                   // optional
+	NewChatMembers        *[]User            `json:"new_chat_members"`        // optional
+	LeftChatMember        *User              `json:"left_chat_member"`        // optional
+	NewChatTitle          string             `json:"new_chat_title"`          // optional
+	NewChatPhoto          *[]PhotoSize       `json:"new_chat_photo"`          // optional
+	DeleteChatPhoto       bool               `json:"delete_chat_photo"`       // optional
+	GroupChatCreated      bool               `json:"group_chat_created"`      // optional
+	SuperGroupChatCreated bool               `json:"supergroup_chat_created"` // optional
+	ChannelChatCreated    bool               `json:"channel_chat_created"`    // optional
+	MigrateToChatID       int64              `json:"migrate_to_chat_id"`      // optional
+	MigrateFromChatID     int64              `json:"migrate_from_chat_id"`    // optional
+	PinnedMessage         *Message           `json:"pinned_message"`          // optional
+	Invoice               *Invoice           `json:"invoice"`                 // optional
+	SuccessfulPayment     *SuccessfulPayment `json:"successful_payment"`      // optional
 }
 
 // Time converts the message timestamp into a Time.
@@ -263,6 +278,15 @@ type Video struct {
 	FileSize  int        `json:"file_size"` // optional
 }
 
+// VideoNote contains information about a video.
+type VideoNote struct {
+	FileID    string     `json:"file_id"`
+	Length    int        `json:"length"`
+	Duration  int        `json:"duration"`
+	Thumbnail *PhotoSize `json:"thumb"`     // optional
+	FileSize  int        `json:"file_size"` // optional
+}
+
 // Voice contains information about a voice.
 type Voice struct {
 	FileID   string `json:"file_id"`
@@ -361,6 +385,7 @@ type InlineKeyboardButton struct {
 	SwitchInlineQuery            *string       `json:"switch_inline_query,omitempty"`              // optional
 	SwitchInlineQueryCurrentChat *string       `json:"switch_inline_query_current_chat,omitempty"` // optional
 	CallbackGame                 *CallbackGame `json:"callback_game,omitempty"`                    // optional
+	Pay                          bool          `json:"pay,omitempty"`			       // optional
 }
 
 // CallbackQuery is data sent when a keyboard button with callback data
@@ -384,8 +409,22 @@ type ForceReply struct {
 
 // ChatMember is information about a member in a chat.
 type ChatMember struct {
-	User   *User  `json:"user"`
-	Status string `json:"status"`
+	User                  *User  `json:"user"`
+	Status                string `json:"status"`
+	UntilDate             int64  `json:"until_date,omitempty"`                // optional
+	CanBeEdited           bool   `json:"can_be_edited,omitempty"`             // optional
+	CanChangeInfo         bool   `json:"can_change_info,omitempty"`           // optional
+	CanPostMessages       bool   `json:"can_post_messages,omitempty"`         // optional
+	CanEditMessages       bool   `json:"can_edit_messages,omitempty"`         // optional
+	CanDeleteMessages     bool   `json:"can_delete_messages,omitempty"`       // optional
+	CanInviteUsers        bool   `json:"can_invite_users,omitempty"`          // optional
+	CanRestrictMembers    bool   `json:"can_restrict_members,omitempty"`      // optional
+	CanPinMessages        bool   `json:"can_pin_messages,omitempty"`          // optional
+	CanPromoteMembers     bool   `json:"can_promote_members,omitempty"`       // optional
+	CanSendMessages       bool   `json:"can_send_messages,omitempty"`         // optional
+	CanSendMediaMessages  bool   `json:"can_send_media_messages,omitempty"`   // optional
+	CanSendOtherMessages  bool   `json:"can_send_other_messages,omitempty"`   // optional
+	CanAddWebPagePreviews bool   `json:"can_add_web_page_previews,omitempty"` // optional
 }
 
 // IsCreator returns if the ChatMember was the creator of the chat.
@@ -493,6 +532,7 @@ type InlineQueryResultGIF struct {
 	URL                 string                `json:"gif_url"` // required
 	Width               int                   `json:"gif_width"`
 	Height              int                   `json:"gif_height"`
+	Duration            int                   `json:"gif_duration"`
 	ThumbURL            string                `json:"thumb_url"`
 	Title               string                `json:"title"`
 	Caption             string                `json:"caption"`
@@ -507,6 +547,7 @@ type InlineQueryResultMPEG4GIF struct {
 	URL                 string                `json:"mpeg4_url"` // required
 	Width               int                   `json:"mpeg4_width"`
 	Height              int                   `json:"mpeg4_height"`
+	Duration            int                   `json:"mpeg4_duration"`
 	ThumbURL            string                `json:"thumb_url"`
 	Title               string                `json:"title"`
 	Caption             string                `json:"caption"`
@@ -635,3 +676,73 @@ type InputContactMessageContent struct {
 	FirstName   string `json:"first_name"`
 	LastName    string `json:"last_name"`
 }
+
+// Invoice contains basic information about an invoice.
+type Invoice struct {
+	Title          string `json:"title"`
+	Description    string `json:"description"`
+	StartParameter string `json:"start_parameter"`
+	Currency       string `json:"currency"`
+	TotalAmount    int    `json:"total_amount"`
+}
+
+// LabeledPrice represents a portion of the price for goods or services.
+type LabeledPrice struct {
+	Label  string `json:"label"`
+	Amount int    `json:"amount"`
+}
+
+// ShippingAddress represents a shipping address.
+type ShippingAddress struct {
+	CountryCode string `json:"country_code"`
+	State       string `json:"state"`
+	City        string `json:"city"`
+	StreetLine1 string `json:"street_line1"`
+	StreetLine2 string `json:"street_line2"`
+	PostCode    string `json:"post_code"`
+}
+
+// OrderInfo represents information about an order.
+type OrderInfo struct {
+	Name            string           `json:"name,omitempty"`
+	PhoneNumber     string           `json:"phone_number,omitempty"`
+	Email           string           `json:"email,omitempty"`
+	ShippingAddress *ShippingAddress `json:"shipping_address,omitempty"`
+}
+
+// ShippingOption represents one shipping option.
+type ShippingOption struct {
+	ID     string          `json:"id"`
+	Title  string          `json:"title"`
+	Prices *[]LabeledPrice `json:"prices"`
+}
+
+// SuccessfulPayment contains basic information about a successful payment.
+type SuccessfulPayment struct {
+	Currency                string     `json:"currency"`
+	TotalAmount             int        `json:"total_amount"`
+	InvoicePayload          string     `json:"invoice_payload"`
+	ShippingOptionID        string     `json:"shipping_option_id,omitempty"`
+	OrderInfo               *OrderInfo `json:"order_info,omitempty"`
+	TelegramPaymentChargeID string     `json:"telegram_payment_charge_id"`
+	ProviderPaymentChargeID string     `json:"provider_payment_charge_id"`
+}
+
+// ShippingQuery contains information about an incoming shipping query.
+type ShippingQuery struct {
+	ID              string           `json:"id"`
+	From            *User            `json:"from"`
+	InvoicePayload  string           `json:"invoice_payload"`
+	ShippingAddress *ShippingAddress `json:"shipping_address"`
+}
+
+// PreCheckoutQuery contains information about an incoming pre-checkout query.
+type PreCheckoutQuery struct {
+	ID               string     `json:"id"`
+	From             *User      `json:"from"`
+	Currency         string     `json:"currency"`
+	TotalAmount      int        `json:"total_amount"`
+	InvoicePayload   string     `json:"invoice_payload"`
+	ShippingOptionID string     `json:"shipping_option_id,omitempty"`
+	OrderInfo        *OrderInfo `json:"order_info,omitempty"`
+}

vendor/github.com/hashicorp/golang-lru/2q.go

@@ -0,0 +1,212 @@
+package lru
+
+import (
+	"fmt"
+	"sync"
+
+	"github.com/hashicorp/golang-lru/simplelru"
+)
+
+const (
+	// Default2QRecentRatio is the ratio of the 2Q cache dedicated
+	// to recently added entries that have only been accessed once.
+	Default2QRecentRatio = 0.25
+
+	// Default2QGhostEntries is the default ratio of ghost
+	// entries kept to track entries recently evicted
+	Default2QGhostEntries = 0.50
+)
+
+// TwoQueueCache is a thread-safe fixed size 2Q cache.
+// 2Q is an enhancement over the standard LRU cache
+// in that it tracks both frequently and recently used
+// entries separately. This avoids a burst in access to new
+// entries from evicting frequently used entries. It adds some
+// additional tracking overhead to the standard LRU cache, and is
+// computationally about 2x the cost, and adds some metadata over
+// head. The ARCCache is similar, but does not require setting any
+// parameters.
+type TwoQueueCache struct {
+	size       int
+	recentSize int
+
+	recent      *simplelru.LRU
+	frequent    *simplelru.LRU
+	recentEvict *simplelru.LRU
+	lock        sync.RWMutex
+}
+
+// New2Q creates a new TwoQueueCache using the default
+// values for the parameters.
+func New2Q(size int) (*TwoQueueCache, error) {
+	return New2QParams(size, Default2QRecentRatio, Default2QGhostEntries)
+}
+
+// New2QParams creates a new TwoQueueCache using the provided
+// parameter values.
+func New2QParams(size int, recentRatio float64, ghostRatio float64) (*TwoQueueCache, error) {
+	if size <= 0 {
+		return nil, fmt.Errorf("invalid size")
+	}
+	if recentRatio < 0.0 || recentRatio > 1.0 {
+		return nil, fmt.Errorf("invalid recent ratio")
+	}
+	if ghostRatio < 0.0 || ghostRatio > 1.0 {
+		return nil, fmt.Errorf("invalid ghost ratio")
+	}
+
+	// Determine the sub-sizes
+	recentSize := int(float64(size) * recentRatio)
+	evictSize := int(float64(size) * ghostRatio)
+
+	// Allocate the LRUs
+	recent, err := simplelru.NewLRU(size, nil)
+	if err != nil {
+		return nil, err
+	}
+	frequent, err := simplelru.NewLRU(size, nil)
+	if err != nil {
+		return nil, err
+	}
+	recentEvict, err := simplelru.NewLRU(evictSize, nil)
+	if err != nil {
+		return nil, err
+	}
+
+	// Initialize the cache
+	c := &TwoQueueCache{
+		size:        size,
+		recentSize:  recentSize,
+		recent:      recent,
+		frequent:    frequent,
+		recentEvict: recentEvict,
+	}
+	return c, nil
+}
+
+func (c *TwoQueueCache) Get(key interface{}) (interface{}, bool) {
+	c.lock.Lock()
+	defer c.lock.Unlock()
+
+	// Check if this is a frequent value
+	if val, ok := c.frequent.Get(key); ok {
+		return val, ok
+	}
+
+	// If the value is contained in recent, then we
+	// promote it to frequent
+	if val, ok := c.recent.Peek(key); ok {
+		c.recent.Remove(key)
+		c.frequent.Add(key, val)
+		return val, ok
+	}
+
+	// No hit
+	return nil, false
+}
+
+func (c *TwoQueueCache) Add(key, value interface{}) {
+	c.lock.Lock()
+	defer c.lock.Unlock()
+
+	// Check if the value is frequently used already,
+	// and just update the value
+	if c.frequent.Contains(key) {
+		c.frequent.Add(key, value)
+		return
+	}
+
+	// Check if the value is recently used, and promote
+	// the value into the frequent list
+	if c.recent.Contains(key) {
+		c.recent.Remove(key)
+		c.frequent.Add(key, value)
+		return
+	}
+
+	// If the value was recently evicted, add it to the
+	// frequently used list
+	if c.recentEvict.Contains(key) {
+		c.ensureSpace(true)
+		c.recentEvict.Remove(key)
+		c.frequent.Add(key, value)
+		return
+	}
+
+	// Add to the recently seen list
+	c.ensureSpace(false)
+	c.recent.Add(key, value)
+	return
+}
+
+// ensureSpace is used to ensure we have space in the cache
+func (c *TwoQueueCache) ensureSpace(recentEvict bool) {
+	// If we have space, nothing to do
+	recentLen := c.recent.Len()
+	freqLen := c.frequent.Len()
+	if recentLen+freqLen < c.size {
+		return
+	}
+
+	// If the recent buffer is larger than
+	// the target, evict from there
+	if recentLen > 0 && (recentLen > c.recentSize || (recentLen == c.recentSize && !recentEvict)) {
+		k, _, _ := c.recent.RemoveOldest()
+		c.recentEvict.Add(k, nil)
+		return
+	}
+
+	// Remove from the frequent list otherwise
+	c.frequent.RemoveOldest()
+}
+
+func (c *TwoQueueCache) Len() int {
+	c.lock.RLock()
+	defer c.lock.RUnlock()
+	return c.recent.Len() + c.frequent.Len()
+}
+
+func (c *TwoQueueCache) Keys() []interface{} {
+	c.lock.RLock()
+	defer c.lock.RUnlock()
+	k1 := c.frequent.Keys()
+	k2 := c.recent.Keys()
+	return append(k1, k2...)
+}
+
+func (c *TwoQueueCache) Remove(key interface{}) {
+	c.lock.Lock()
+	defer c.lock.Unlock()
+	if c.frequent.Remove(key) {
+		return
+	}
+	if c.recent.Remove(key) {
+		return
+	}
+	if c.recentEvict.Remove(key) {
+		return
+	}
+}
+
+func (c *TwoQueueCache) Purge() {
+	c.lock.Lock()
+	defer c.lock.Unlock()
+	c.recent.Purge()
+	c.frequent.Purge()
+	c.recentEvict.Purge()
+}
+
+func (c *TwoQueueCache) Contains(key interface{}) bool {
+	c.lock.RLock()
+	defer c.lock.RUnlock()
+	return c.frequent.Contains(key) || c.recent.Contains(key)
+}
+
+func (c *TwoQueueCache) Peek(key interface{}) (interface{}, bool) {
+	c.lock.RLock()
+	defer c.lock.RUnlock()
+	if val, ok := c.frequent.Peek(key); ok {
+		return val, ok
+	}
+	return c.recent.Peek(key)
+}

vendor/github.com/hashicorp/golang-lru/LICENSE

@@ -0,0 +1,362 @@
+Mozilla Public License, version 2.0
+
+1. Definitions
+
+1.1. "Contributor"
+
+     means each individual or legal entity that creates, contributes to the
+     creation of, or owns Covered Software.
+
+1.2. "Contributor Version"
+
+     means the combination of the Contributions of others (if any) used by a
+     Contributor and that particular Contributor's Contribution.
+
+1.3. "Contribution"
+
+     means Covered Software of a particular Contributor.
+
+1.4. "Covered Software"
+
+     means Source Code Form to which the initial Contributor has attached the
+     notice in Exhibit A, the Executable Form of such Source Code Form, and
+     Modifications of such Source Code Form, in each case including portions
+     thereof.
+
+1.5. "Incompatible With Secondary Licenses"
+     means
+
+     a. that the initial Contributor has attached the notice described in
+        Exhibit B to the Covered Software; or
+
+     b. that the Covered Software was made available under the terms of
+        version 1.1 or earlier of the License, but not also under the terms of
+        a Secondary License.
+
+1.6. "Executable Form"
+
+     means any form of the work other than Source Code Form.
+
+1.7. "Larger Work"
+
+     means a work that combines Covered Software with other material, in a
+     separate file or files, that is not Covered Software.
+
+1.8. "License"
+
+     means this document.
+
+1.9. "Licensable"
+
+     means having the right to grant, to the maximum extent possible, whether
+     at the time of the initial grant or subsequently, any and all of the
+     rights conveyed by this License.
+
+1.10. "Modifications"
+
+     means any of the following:
+
+     a. any file in Source Code Form that results from an addition to,
+        deletion from, or modification of the contents of Covered Software; or
+
+     b. any new file in Source Code Form that contains any Covered Software.
+
+1.11. "Patent Claims" of a Contributor
+
+      means any patent claim(s), including without limitation, method,
+      process, and apparatus claims, in any patent Licensable by such
+      Contributor that would be infringed, but for the grant of the License,
+      by the making, using, selling, offering for sale, having made, import,
+      or transfer of either its Contributions or its Contributor Version.
+
+1.12. "Secondary License"
+
+      means either the GNU General Public License, Version 2.0, the GNU Lesser
+      General Public License, Version 2.1, the GNU Affero General Public
+      License, Version 3.0, or any later versions of those licenses.
+
+1.13. "Source Code Form"
+
+      means the form of the work preferred for making modifications.
+
+1.14. "You" (or "Your")
+
+      means an individual or a legal entity exercising rights under this
+      License. For legal entities, "You" includes any entity that controls, is
+      controlled by, or is under common control with You. For purposes of this
+      definition, "control" means (a) the power, direct or indirect, to cause
+      the direction or management of such entity, whether by contract or
+      otherwise, or (b) ownership of more than fifty percent (50%) of the
+      outstanding shares or beneficial ownership of such entity.
+
+
+2. License Grants and Conditions
+
+2.1. Grants
+
+     Each Contributor hereby grants You a world-wide, royalty-free,
+     non-exclusive license:
+
+     a. under intellectual property rights (other than patent or trademark)
+        Licensable by such Contributor to use, reproduce, make available,
+        modify, display, perform, distribute, and otherwise exploit its
+        Contributions, either on an unmodified basis, with Modifications, or
+        as part of a Larger Work; and
+
+     b. under Patent Claims of such Contributor to make, use, sell, offer for
+        sale, have made, import, and otherwise transfer either its
+        Contributions or its Contributor Version.
+
+2.2. Effective Date
+
+     The licenses granted in Section 2.1 with respect to any Contribution
+     become effective for each Contribution on the date the Contributor first
+     distributes such Contribution.
+
+2.3. Limitations on Grant Scope
+
+     The licenses granted in this Section 2 are the only rights granted under
+     this License. No additional rights or licenses will be implied from the
+     distribution or licensing of Covered Software under this License.
+     Notwithstanding Section 2.1(b) above, no patent license is granted by a
+     Contributor:
+
+     a. for any code that a Contributor has removed from Covered Software; or
+
+     b. for infringements caused by: (i) Your and any other third party's
+        modifications of Covered Software, or (ii) the combination of its
+        Contributions with other software (except as part of its Contributor
+        Version); or
+
+     c. under Patent Claims infringed by Covered Software in the absence of
+        its Contributions.
+
+     This License does not grant any rights in the trademarks, service marks,
+     or logos of any Contributor (except as may be necessary to comply with
+     the notice requirements in Section 3.4).
+
+2.4. Subsequent Licenses
+
+     No Contributor makes additional grants as a result of Your choice to
+     distribute the Covered Software under a subsequent version of this
+     License (see Section 10.2) or under the terms of a Secondary License (if
+     permitted under the terms of Section 3.3).
+
+2.5. Representation
+
+     Each Contributor represents that the Contributor believes its
+     Contributions are its original creation(s) or it has sufficient rights to
+     grant the rights to its Contributions conveyed by this License.
+
+2.6. Fair Use
+
+     This License is not intended to limit any rights You have under
+     applicable copyright doctrines of fair use, fair dealing, or other
+     equivalents.
+
+2.7. Conditions
+
+     Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted in
+     Section 2.1.
+
+
+3. Responsibilities
+
+3.1. Distribution of Source Form
+
+     All distribution of Covered Software in Source Code Form, including any
+     Modifications that You create or to which You contribute, must be under
+     the terms of this License. You must inform recipients that the Source
+     Code Form of the Covered Software is governed by the terms of this
+     License, and how they can obtain a copy of this License. You may not
+     attempt to alter or restrict the recipients' rights in the Source Code
+     Form.
+
+3.2. Distribution of Executable Form
+
+     If You distribute Covered Software in Executable Form then:
+
+     a. such Covered Software must also be made available in Source Code Form,
+        as described in Section 3.1, and You must inform recipients of the
+        Executable Form how they can obtain a copy of such Source Code Form by
+        reasonable means in a timely manner, at a charge no more than the cost
+        of distribution to the recipient; and
+
+     b. You may distribute such Executable Form under the terms of this
+        License, or sublicense it under different terms, provided that the
+        license for the Executable Form does not attempt to limit or alter the
+        recipients' rights in the Source Code Form under this License.
+
+3.3. Distribution of a Larger Work
+
+     You may create and distribute a Larger Work under terms of Your choice,
+     provided that You also comply with the requirements of this License for
+     the Covered Software. If the Larger Work is a combination of Covered
+     Software with a work governed by one or more Secondary Licenses, and the
+     Covered Software is not Incompatible With Secondary Licenses, this
+     License permits You to additionally distribute such Covered Software
+     under the terms of such Secondary License(s), so that the recipient of
+     the Larger Work may, at their option, further distribute the Covered
+     Software under the terms of either this License or such Secondary
+     License(s).
+
+3.4. Notices
+
+     You may not remove or alter the substance of any license notices
+     (including copyright notices, patent notices, disclaimers of warranty, or
+     limitations of liability) contained within the Source Code Form of the
+     Covered Software, except that You may alter any license notices to the
+     extent required to remedy known factual inaccuracies.
+
+3.5. Application of Additional Terms
+
+     You may choose to offer, and to charge a fee for, warranty, support,
+     indemnity or liability obligations to one or more recipients of Covered
+     Software. However, You may do so only on Your own behalf, and not on
+     behalf of any Contributor. You must make it absolutely clear that any
+     such warranty, support, indemnity, or liability obligation is offered by
+     You alone, and You hereby agree to indemnify every Contributor for any
+     liability incurred by such Contributor as a result of warranty, support,
+     indemnity or liability terms You offer. You may include additional
+     disclaimers of warranty and limitations of liability specific to any
+     jurisdiction.
+
+4. Inability to Comply Due to Statute or Regulation
+
+   If it is impossible for You to comply with any of the terms of this License
+   with respect to some or all of the Covered Software due to statute,
+   judicial order, or regulation then You must: (a) comply with the terms of
+   this License to the maximum extent possible; and (b) describe the
+   limitations and the code they affect. Such description must be placed in a
+   text file included with all distributions of the Covered Software under
+   this License. Except to the extent prohibited by statute or regulation,
+   such description must be sufficiently detailed for a recipient of ordinary
+   skill to be able to understand it.
+
+5. Termination
+
+5.1. The rights granted under this License will terminate automatically if You
+     fail to comply with any of its terms. However, if You become compliant,
+     then the rights granted under this License from a particular Contributor
+     are reinstated (a) provisionally, unless and until such Contributor
+     explicitly and finally terminates Your grants, and (b) on an ongoing
+     basis, if such Contributor fails to notify You of the non-compliance by
+     some reasonable means prior to 60 days after You have come back into
+     compliance. Moreover, Your grants from a particular Contributor are
+     reinstated on an ongoing basis if such Contributor notifies You of the
+     non-compliance by some reasonable means, this is the first time You have
+     received notice of non-compliance with this License from such
+     Contributor, and You become compliant prior to 30 days after Your receipt
+     of the notice.
+
+5.2. If You initiate litigation against any entity by asserting a patent
+     infringement claim (excluding declaratory judgment actions,
+     counter-claims, and cross-claims) alleging that a Contributor Version
+     directly or indirectly infringes any patent, then the rights granted to
+     You by any and all Contributors for the Covered Software under Section
+     2.1 of this License shall terminate.
+
+5.3. In the event of termination under Sections 5.1 or 5.2 above, all end user
+     license agreements (excluding distributors and resellers) which have been
+     validly granted by You or Your distributors under this License prior to
+     termination shall survive termination.
+
+6. Disclaimer of Warranty
+
+   Covered Software is provided under this License on an "as is" basis,
+   without warranty of any kind, either expressed, implied, or statutory,
+   including, without limitation, warranties that the Covered Software is free
+   of defects, merchantable, fit for a particular purpose or non-infringing.
+   The entire risk as to the quality and performance of the Covered Software
+   is with You. Should any Covered Software prove defective in any respect,
+   You (not any Contributor) assume the cost of any necessary servicing,
+   repair, or correction. This disclaimer of warranty constitutes an essential
+   part of this License. No use of  any Covered Software is authorized under
+   this License except under this disclaimer.
+
+7. Limitation of Liability
+
+   Under no circumstances and under no legal theory, whether tort (including
+   negligence), contract, or otherwise, shall any Contributor, or anyone who
+   distributes Covered Software as permitted above, be liable to You for any
+   direct, indirect, special, incidental, or consequential damages of any
+   character including, without limitation, damages for lost profits, loss of
+   goodwill, work stoppage, computer failure or malfunction, or any and all
+   other commercial damages or losses, even if such party shall have been
+   informed of the possibility of such damages. This limitation of liability
+   shall not apply to liability for death or personal injury resulting from
+   such party's negligence to the extent applicable law prohibits such
+   limitation. Some jurisdictions do not allow the exclusion or limitation of
+   incidental or consequential damages, so this exclusion and limitation may
+   not apply to You.
+
+8. Litigation
+
+   Any litigation relating to this License may be brought only in the courts
+   of a jurisdiction where the defendant maintains its principal place of
+   business and such litigation shall be governed by laws of that
+   jurisdiction, without reference to its conflict-of-law provisions. Nothing
+   in this Section shall prevent a party's ability to bring cross-claims or
+   counter-claims.
+
+9. Miscellaneous
+
+   This License represents the complete agreement concerning the subject
+   matter hereof. If any provision of this License is held to be
+   unenforceable, such provision shall be reformed only to the extent
+   necessary to make it enforceable. Any law or regulation which provides that
+   the language of a contract shall be construed against the drafter shall not
+   be used to construe this License against a Contributor.
+
+
+10. Versions of the License
+
+10.1. New Versions
+
+      Mozilla Foundation is the license steward. Except as provided in Section
+      10.3, no one other than the license steward has the right to modify or
+      publish new versions of this License. Each version will be given a
+      distinguishing version number.
+
+10.2. Effect of New Versions
+
+      You may distribute the Covered Software under the terms of the version
+      of the License under which You originally received the Covered Software,
+      or under the terms of any subsequent version published by the license
+      steward.
+
+10.3. Modified Versions
+
+      If you create software not governed by this License, and you want to
+      create a new license for such software, you may create and use a
+      modified version of this License if you rename the license and remove
+      any references to the name of the license steward (except to note that
+      such modified license differs from this License).
+
+10.4. Distributing Source Code Form that is Incompatible With Secondary
+      Licenses If You choose to distribute Source Code Form that is
+      Incompatible With Secondary Licenses under the terms of this version of
+      the License, the notice described in Exhibit B of this License must be
+      attached.
+
+Exhibit A - Source Code Form License Notice
+
+      This Source Code Form is subject to the
+      terms of the Mozilla Public License, v.
+      2.0. If a copy of the MPL was not
+      distributed with this file, You can
+      obtain one at
+      http://mozilla.org/MPL/2.0/.
+
+If it is not possible or desirable to put the notice in a particular file,
+then You may include the notice in a location (such as a LICENSE file in a
+relevant directory) where a recipient would be likely to look for such a
+notice.
+
+You may add additional accurate notices of copyright ownership.
+
+Exhibit B - "Incompatible With Secondary Licenses" Notice
+
+      This Source Code Form is "Incompatible
+      With Secondary Licenses", as defined by
+      the Mozilla Public License, v. 2.0.

vendor/github.com/hashicorp/golang-lru/arc.go

@@ -0,0 +1,257 @@
+package lru
+
+import (
+	"sync"
+
+	"github.com/hashicorp/golang-lru/simplelru"
+)
+
+// ARCCache is a thread-safe fixed size Adaptive Replacement Cache (ARC).
+// ARC is an enhancement over the standard LRU cache in that tracks both
+// frequency and recency of use. This avoids a burst in access to new
+// entries from evicting the frequently used older entries. It adds some
+// additional tracking overhead to a standard LRU cache, computationally
+// it is roughly 2x the cost, and the extra memory overhead is linear
+// with the size of the cache. ARC has been patented by IBM, but is
+// similar to the TwoQueueCache (2Q) which requires setting parameters.
+type ARCCache struct {
+	size int // Size is the total capacity of the cache
+	p    int // P is the dynamic preference towards T1 or T2
+
+	t1 *simplelru.LRU // T1 is the LRU for recently accessed items
+	b1 *simplelru.LRU // B1 is the LRU for evictions from t1
+
+	t2 *simplelru.LRU // T2 is the LRU for frequently accessed items
+	b2 *simplelru.LRU // B2 is the LRU for evictions from t2
+
+	lock sync.RWMutex
+}
+
+// NewARC creates an ARC of the given size
+func NewARC(size int) (*ARCCache, error) {
+	// Create the sub LRUs
+	b1, err := simplelru.NewLRU(size, nil)
+	if err != nil {
+		return nil, err
+	}
+	b2, err := simplelru.NewLRU(size, nil)
+	if err != nil {
+		return nil, err
+	}
+	t1, err := simplelru.NewLRU(size, nil)
+	if err != nil {
+		return nil, err
+	}
+	t2, err := simplelru.NewLRU(size, nil)
+	if err != nil {
+		return nil, err
+	}
+
+	// Initialize the ARC
+	c := &ARCCache{
+		size: size,
+		p:    0,
+		t1:   t1,
+		b1:   b1,
+		t2:   t2,
+		b2:   b2,
+	}
+	return c, nil
+}
+
+// Get looks up a key's value from the cache.
+func (c *ARCCache) Get(key interface{}) (interface{}, bool) {
+	c.lock.Lock()
+	defer c.lock.Unlock()
+
+	// Ff the value is contained in T1 (recent), then
+	// promote it to T2 (frequent)
+	if val, ok := c.t1.Peek(key); ok {
+		c.t1.Remove(key)
+		c.t2.Add(key, val)
+		return val, ok
+	}
+
+	// Check if the value is contained in T2 (frequent)
+	if val, ok := c.t2.Get(key); ok {
+		return val, ok
+	}
+
+	// No hit
+	return nil, false
+}
+
+// Add adds a value to the cache.
+func (c *ARCCache) Add(key, value interface{}) {
+	c.lock.Lock()
+	defer c.lock.Unlock()
+
+	// Check if the value is contained in T1 (recent), and potentially
+	// promote it to frequent T2
+	if c.t1.Contains(key) {
+		c.t1.Remove(key)
+		c.t2.Add(key, value)
+		return
+	}
+
+	// Check if the value is already in T2 (frequent) and update it
+	if c.t2.Contains(key) {
+		c.t2.Add(key, value)
+		return
+	}
+
+	// Check if this value was recently evicted as part of the
+	// recently used list
+	if c.b1.Contains(key) {
+		// T1 set is too small, increase P appropriately
+		delta := 1
+		b1Len := c.b1.Len()
+		b2Len := c.b2.Len()
+		if b2Len > b1Len {
+			delta = b2Len / b1Len
+		}
+		if c.p+delta >= c.size {
+			c.p = c.size
+		} else {
+			c.p += delta
+		}
+
+		// Potentially need to make room in the cache
+		if c.t1.Len()+c.t2.Len() >= c.size {
+			c.replace(false)
+		}
+
+		// Remove from B1
+		c.b1.Remove(key)
+
+		// Add the key to the frequently used list
+		c.t2.Add(key, value)
+		return
+	}
+
+	// Check if this value was recently evicted as part of the
+	// frequently used list
+	if c.b2.Contains(key) {
+		// T2 set is too small, decrease P appropriately
+		delta := 1
+		b1Len := c.b1.Len()
+		b2Len := c.b2.Len()
+		if b1Len > b2Len {
+			delta = b1Len / b2Len
+		}
+		if delta >= c.p {
+			c.p = 0
+		} else {
+			c.p -= delta
+		}
+
+		// Potentially need to make room in the cache
+		if c.t1.Len()+c.t2.Len() >= c.size {
+			c.replace(true)
+		}
+
+		// Remove from B2
+		c.b2.Remove(key)
+
+		// Add the key to the frequntly used list
+		c.t2.Add(key, value)
+		return
+	}
+
+	// Potentially need to make room in the cache
+	if c.t1.Len()+c.t2.Len() >= c.size {
+		c.replace(false)
+	}
+
+	// Keep the size of the ghost buffers trim
+	if c.b1.Len() > c.size-c.p {
+		c.b1.RemoveOldest()
+	}
+	if c.b2.Len() > c.p {
+		c.b2.RemoveOldest()
+	}
+
+	// Add to the recently seen list
+	c.t1.Add(key, value)
+	return
+}
+
+// replace is used to adaptively evict from either T1 or T2
+// based on the current learned value of P
+func (c *ARCCache) replace(b2ContainsKey bool) {
+	t1Len := c.t1.Len()
+	if t1Len > 0 && (t1Len > c.p || (t1Len == c.p && b2ContainsKey)) {
+		k, _, ok := c.t1.RemoveOldest()
+		if ok {
+			c.b1.Add(k, nil)
+		}
+	} else {
+		k, _, ok := c.t2.RemoveOldest()
+		if ok {
+			c.b2.Add(k, nil)
+		}
+	}
+}
+
+// Len returns the number of cached entries
+func (c *ARCCache) Len() int {
+	c.lock.RLock()
+	defer c.lock.RUnlock()
+	return c.t1.Len() + c.t2.Len()
+}
+
+// Keys returns all the cached keys
+func (c *ARCCache) Keys() []interface{} {
+	c.lock.RLock()
+	defer c.lock.RUnlock()
+	k1 := c.t1.Keys()
+	k2 := c.t2.Keys()
+	return append(k1, k2...)
+}
+
+// Remove is used to purge a key from the cache
+func (c *ARCCache) Remove(key interface{}) {
+	c.lock.Lock()
+	defer c.lock.Unlock()
+	if c.t1.Remove(key) {
+		return
+	}
+	if c.t2.Remove(key) {
+		return
+	}
+	if c.b1.Remove(key) {
+		return
+	}
+	if c.b2.Remove(key) {
+		return
+	}
+}
+
+// Purge is used to clear the cache
+func (c *ARCCache) Purge() {
+	c.lock.Lock()
+	defer c.lock.Unlock()
+	c.t1.Purge()
+	c.t2.Purge()
+	c.b1.Purge()
+	c.b2.Purge()
+}
+
+// Contains is used to check if the cache contains a key
+// without updating recency or frequency.
+func (c *ARCCache) Contains(key interface{}) bool {
+	c.lock.RLock()
+	defer c.lock.RUnlock()
+	return c.t1.Contains(key) || c.t2.Contains(key)
+}
+
+// Peek is used to inspect the cache value of a key
+// without updating recency or frequency.
+func (c *ARCCache) Peek(key interface{}) (interface{}, bool) {
+	c.lock.RLock()
+	defer c.lock.RUnlock()
+	if val, ok := c.t1.Peek(key); ok {
+		return val, ok
+	}
+	return c.t2.Peek(key)
+}

vendor/github.com/hashicorp/golang-lru/lru.go

@@ -0,0 +1,114 @@
+// This package provides a simple LRU cache. It is based on the
+// LRU implementation in groupcache:
+// https://github.com/golang/groupcache/tree/master/lru
+package lru
+
+import (
+	"sync"
+
+	"github.com/hashicorp/golang-lru/simplelru"
+)
+
+// Cache is a thread-safe fixed size LRU cache.
+type Cache struct {
+	lru  *simplelru.LRU
+	lock sync.RWMutex
+}
+
+// New creates an LRU of the given size
+func New(size int) (*Cache, error) {
+	return NewWithEvict(size, nil)
+}
+
+// NewWithEvict constructs a fixed size cache with the given eviction
+// callback.
+func NewWithEvict(size int, onEvicted func(key interface{}, value interface{})) (*Cache, error) {
+	lru, err := simplelru.NewLRU(size, simplelru.EvictCallback(onEvicted))
+	if err != nil {
+		return nil, err
+	}
+	c := &Cache{
+		lru: lru,
+	}
+	return c, nil
+}
+
+// Purge is used to completely clear the cache
+func (c *Cache) Purge() {
+	c.lock.Lock()
+	c.lru.Purge()
+	c.lock.Unlock()
+}
+
+// Add adds a value to the cache.  Returns true if an eviction occurred.
+func (c *Cache) Add(key, value interface{}) bool {
+	c.lock.Lock()
+	defer c.lock.Unlock()
+	return c.lru.Add(key, value)
+}
+
+// Get looks up a key's value from the cache.
+func (c *Cache) Get(key interface{}) (interface{}, bool) {
+	c.lock.Lock()
+	defer c.lock.Unlock()
+	return c.lru.Get(key)
+}
+
+// Check if a key is in the cache, without updating the recent-ness
+// or deleting it for being stale.
+func (c *Cache) Contains(key interface{}) bool {
+	c.lock.RLock()
+	defer c.lock.RUnlock()
+	return c.lru.Contains(key)
+}
+
+// Returns the key value (or undefined if not found) without updating
+// the "recently used"-ness of the key.
+func (c *Cache) Peek(key interface{}) (interface{}, bool) {
+	c.lock.RLock()
+	defer c.lock.RUnlock()
+	return c.lru.Peek(key)
+}
+
+// ContainsOrAdd checks if a key is in the cache  without updating the
+// recent-ness or deleting it for being stale,  and if not, adds the value.
+// Returns whether found and whether an eviction occurred.
+func (c *Cache) ContainsOrAdd(key, value interface{}) (ok, evict bool) {
+	c.lock.Lock()
+	defer c.lock.Unlock()
+
+	if c.lru.Contains(key) {
+		return true, false
+	} else {
+		evict := c.lru.Add(key, value)
+		return false, evict
+	}
+}
+
+// Remove removes the provided key from the cache.
+func (c *Cache) Remove(key interface{}) {
+	c.lock.Lock()
+	c.lru.Remove(key)
+	c.lock.Unlock()
+}
+
+// RemoveOldest removes the oldest item from the cache.
+func (c *Cache) RemoveOldest() {
+	c.lock.Lock()
+	c.lru.RemoveOldest()
+	c.lock.Unlock()
+}
+
+// Keys returns a slice of the keys in the cache, from oldest to newest.
+func (c *Cache) Keys() []interface{} {
+	c.lock.RLock()
+	defer c.lock.RUnlock()
+	return c.lru.Keys()
+}
+
+// Len returns the number of items in the cache.
+func (c *Cache) Len() int {
+	c.lock.RLock()
+	defer c.lock.RUnlock()
+	return c.lru.Len()
+}

vendor/github.com/hashicorp/golang-lru/simplelru/lru.go

@@ -0,0 +1,160 @@
+package simplelru
+
+import (
+	"container/list"
+	"errors"
+)
+
+// EvictCallback is used to get a callback when a cache entry is evicted
+type EvictCallback func(key interface{}, value interface{})
+
+// LRU implements a non-thread safe fixed size LRU cache
+type LRU struct {
+	size      int
+	evictList *list.List
+	items     map[interface{}]*list.Element
+	onEvict   EvictCallback
+}
+
+// entry is used to hold a value in the evictList
+type entry struct {
+	key   interface{}
+	value interface{}
+}
+
+// NewLRU constructs an LRU of the given size
+func NewLRU(size int, onEvict EvictCallback) (*LRU, error) {
+	if size <= 0 {
+		return nil, errors.New("Must provide a positive size")
+	}
+	c := &LRU{
+		size:      size,
+		evictList: list.New(),
+		items:     make(map[interface{}]*list.Element),
+		onEvict:   onEvict,
+	}
+	return c, nil
+}
+
+// Purge is used to completely clear the cache
+func (c *LRU) Purge() {
+	for k, v := range c.items {
+		if c.onEvict != nil {
+			c.onEvict(k, v.Value.(*entry).value)
+		}
+		delete(c.items, k)
+	}
+	c.evictList.Init()
+}
+
+// Add adds a value to the cache.  Returns true if an eviction occurred.
+func (c *LRU) Add(key, value interface{}) bool {
+	// Check for existing item
+	if ent, ok := c.items[key]; ok {
+		c.evictList.MoveToFront(ent)
+		ent.Value.(*entry).value = value
+		return false
+	}
+
+	// Add new item
+	ent := &entry{key, value}
+	entry := c.evictList.PushFront(ent)
+	c.items[key] = entry
+
+	evict := c.evictList.Len() > c.size
+	// Verify size not exceeded
+	if evict {
+		c.removeOldest()
+	}
+	return evict
+}
+
+// Get looks up a key's value from the cache.
+func (c *LRU) Get(key interface{}) (value interface{}, ok bool) {
+	if ent, ok := c.items[key]; ok {
+		c.evictList.MoveToFront(ent)
+		return ent.Value.(*entry).value, true
+	}
+	return
+}
+
+// Check if a key is in the cache, without updating the recent-ness
+// or deleting it for being stale.
+func (c *LRU) Contains(key interface{}) (ok bool) {
+	_, ok = c.items[key]
+	return ok
+}
+
+// Returns the key value (or undefined if not found) without updating
+// the "recently used"-ness of the key.
+func (c *LRU) Peek(key interface{}) (value interface{}, ok bool) {
+	if ent, ok := c.items[key]; ok {
+		return ent.Value.(*entry).value, true
+	}
+	return nil, ok
+}
+
+// Remove removes the provided key from the cache, returning if the
+// key was contained.
+func (c *LRU) Remove(key interface{}) bool {
+	if ent, ok := c.items[key]; ok {
+		c.removeElement(ent)
+		return true
+	}
+	return false
+}
+
+// RemoveOldest removes the oldest item from the cache.
+func (c *LRU) RemoveOldest() (interface{}, interface{}, bool) {
+	ent := c.evictList.Back()
+	if ent != nil {
+		c.removeElement(ent)
+		kv := ent.Value.(*entry)
+		return kv.key, kv.value, true
+	}
+	return nil, nil, false
+}
+
+// GetOldest returns the oldest entry
+func (c *LRU) GetOldest() (interface{}, interface{}, bool) {
+	ent := c.evictList.Back()
+	if ent != nil {
+		kv := ent.Value.(*entry)
+		return kv.key, kv.value, true
+	}
+	return nil, nil, false
+}
+
+// Keys returns a slice of the keys in the cache, from oldest to newest.
+func (c *LRU) Keys() []interface{} {
+	keys := make([]interface{}, len(c.items))
+	i := 0
+	for ent := c.evictList.Back(); ent != nil; ent = ent.Prev() {
+		keys[i] = ent.Value.(*entry).key
+		i++
+	}
+	return keys
+}
+
+// Len returns the number of items in the cache.
+func (c *LRU) Len() int {
+	return c.evictList.Len()
+}
+
+// removeOldest removes the oldest item from the cache.
+func (c *LRU) removeOldest() {
+	ent := c.evictList.Back()
+	if ent != nil {
+		c.removeElement(ent)
+	}
+}
+
+// removeElement is used to remove a given list element from the cache
+func (c *LRU) removeElement(e *list.Element) {
+	c.evictList.Remove(e)
+	kv := e.Value.(*entry)
+	delete(c.items, kv.key)
+	if c.onEvict != nil {
+		c.onEvict(kv.key, kv.value)
+	}
+}

vendor/github.com/mattermost/platform/einterfaces/LICENSE.txt

@@ -1,32 +1,32 @@
 Mattermost Licensing
 
-SOFTWARE LICENSING 
+SOFTWARE LICENSING
 
-You are licensed to use compiled versions of the Mattermost platform produced by Mattermost, Inc. under an MIT LICENSE 
+You are licensed to use compiled versions of the Mattermost platform produced by Mattermost, Inc. under an MIT LICENSE
 
 -	See MIT-COMPILED-LICENSE.md included in compiled versions for details
 
 You may be licensed to use source code to create compiled versions not produced by Mattermost, Inc. in one of two ways:
 
-1. Under the Free Software Foundation’s GNU AGPL v.3.0, subject to the exceptions outlined in this policy; or 
-2. Under a commercial license available from Mattermost, Inc. by contacting commercial@mattermost.com 
+1. Under the Free Software Foundation’s GNU AGPL v.3.0, subject to the exceptions outlined in this policy; or
+2. Under a commercial license available from Mattermost, Inc. by contacting commercial@mattermost.com
 
-You are licensed to use the source code in Admin Tools and Configuration Files (templates/, config/, model/, 
+You are licensed to use the source code in Admin Tools and Configuration Files (templates/, config/, model/,
 webapp/client, webapp/fonts, webapp/i18n, webapp/images and all subdirectories thereof) under the Apache License v2.0.
 
-We promise that we will not enforce the copyleft provisions in AGPL v3.0 against you if your application (a) does not 
+We promise that we will not enforce the copyleft provisions in AGPL v3.0 against you if your application (a) does not
 link to the Mattermost Platform directly, but exclusively uses the Mattermost Admin Tools and Configuration Files, and
-(b) you have not modified, added to or adapted the source code of Mattermost in a way that results in the creation of 
+(b) you have not modified, added to or adapted the source code of Mattermost in a way that results in the creation of
 a “modified version” or “work based on” Mattermost as these terms are defined in the AGPL v3.0 license.
 
 MATTERMOST TRADEMARK GUIDELINES
 
-Your use of the mark Mattermost is subject to Mattermost, Inc's prior written approval and our organization’s Trademark 
-Standards of Use at http://www.mattermost.org/trademark-standards-of-use/. For trademark approval or any questions 
-you have about using these trademarks, please email trademark@mattermost.com 
+Your use of the mark Mattermost is subject to Mattermost, Inc's prior written approval and our organization’s Trademark
+Standards of Use at http://www.mattermost.org/trademark-standards-of-use/. For trademark approval or any questions
+you have about using these trademarks, please email trademark@mattermost.com
 
 ------------------------------------------------------------------------------------------------------------------------------
-                               
+
                                Apache License
                            Version 2.0, January 2004
                         http://www.apache.org/licenses/

vendor/github.com/mattermost/platform/einterfaces/account_migration.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2016 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2016-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package einterfaces
@@ -6,7 +6,7 @@ package einterfaces
 import "github.com/mattermost/platform/model"
 
 type AccountMigrationInterface interface {
-	MigrateToLdap(fromAuthService string, forignUserFieldNameToMatch string) *model.AppError
+	MigrateToLdap(fromAuthService string, forignUserFieldNameToMatch string, force bool) *model.AppError
 }
 
 var theAccountMigrationInterface AccountMigrationInterface

vendor/github.com/mattermost/platform/einterfaces/brand.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2016 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2016-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package einterfaces

vendor/github.com/mattermost/platform/einterfaces/cluster.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2015 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package einterfaces
@@ -7,26 +7,19 @@ import (
 	"github.com/mattermost/platform/model"
 )
 
+type ClusterMessageHandler func(msg *model.ClusterMessage)
+
 type ClusterInterface interface {
 	StartInterNodeCommunication()
 	StopInterNodeCommunication()
-	GetClusterInfos() []*model.ClusterInfo
-	GetClusterStats() ([]*model.ClusterStats, *model.AppError)
-	ClearSessionCacheForUser(userId string)
-	InvalidateCacheForUser(userId string)
-	InvalidateCacheForChannel(channelId string)
-	InvalidateCacheForChannelByName(teamId, name string)
-	InvalidateCacheForChannelMembers(channelId string)
-	InvalidateCacheForChannelMembersNotifyProps(channelId string)
-	InvalidateCacheForChannelPosts(channelId string)
-	InvalidateCacheForWebhook(webhookId string)
-	InvalidateCacheForReactions(postId string)
-	Publish(event *model.WebSocketEvent)
-	UpdateStatus(status *model.Status)
-	GetLogs() ([]string, *model.AppError)
+	RegisterClusterMessageHandler(event string, crm ClusterMessageHandler)
 	GetClusterId() string
+	GetClusterInfos() []*model.ClusterInfo
+	SendClusterMessage(cluster *model.ClusterMessage)
+	NotifyMsg(buf []byte)
+	GetClusterStats() ([]*model.ClusterStats, *model.AppError)
+	GetLogs(page, perPage int) ([]string, *model.AppError)
 	ConfigChanged(previousConfig *model.Config, newConfig *model.Config, sendToOtherServer bool) *model.AppError
-	InvalidateAllCaches() *model.AppError
 }
 
 var theClusterInterface ClusterInterface

vendor/github.com/mattermost/platform/einterfaces/compliance.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2015 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package einterfaces

vendor/github.com/mattermost/platform/einterfaces/elasticsearch.go

@@ -0,0 +1,25 @@
+// Copyright (c) 2017-present Mattermost, Inc. All Rights Reserved.
+// See License.txt for license information.
+
+package einterfaces
+
+import "github.com/mattermost/platform/model"
+
+type ElasticsearchInterface interface {
+	Start() *model.AppError
+	IndexPost(post *model.Post, teamId string) *model.AppError
+	SearchPosts(channels *model.ChannelList, searchParams []*model.SearchParams) ([]string, *model.AppError)
+	DeletePost(post *model.Post) *model.AppError
+	TestConfig(cfg *model.Config) *model.AppError
+	PurgeIndexes() *model.AppError
+}
+
+var theElasticsearchInterface ElasticsearchInterface
+
+func RegisterElasticsearchInterface(newInterface ElasticsearchInterface) {
+	theElasticsearchInterface = newInterface
+}
+
+func GetElasticsearchInterface() ElasticsearchInterface {
+	return theElasticsearchInterface
+}

vendor/github.com/mattermost/platform/einterfaces/emoji.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2016 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2016-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package einterfaces

vendor/github.com/mattermost/platform/einterfaces/jobs/data_retention.go

@@ -0,0 +1,23 @@
+// Copyright (c) 2017-present Mattermost, Inc. All Rights Reserved.
+// See License.txt for license information.
+
+package jobs
+
+import (
+	"github.com/mattermost/platform/model"
+)
+
+type DataRetentionInterface interface {
+	MakeWorker() model.Worker
+	MakeScheduler() model.Scheduler
+}
+
+var theDataRetentionInterface DataRetentionInterface
+
+func RegisterDataRetentionInterface(newInterface DataRetentionInterface) {
+	theDataRetentionInterface = newInterface
+}
+
+func GetDataRetentionInterface() DataRetentionInterface {
+	return theDataRetentionInterface
+}

vendor/github.com/mattermost/platform/einterfaces/jobs/elasticsearch.go

@@ -0,0 +1,22 @@
+// Copyright (c) 2017-present Mattermost, Inc. All Rights Reserved.
+// See License.txt for license information.
+
+package jobs
+
+import (
+	"github.com/mattermost/platform/model"
+)
+
+type ElasticsearchIndexerInterface interface {
+	MakeWorker() model.Worker
+}
+
+var theElasticsearchIndexerInterface ElasticsearchIndexerInterface
+
+func RegisterElasticsearchIndexerInterface(newInterface ElasticsearchIndexerInterface) {
+	theElasticsearchIndexerInterface = newInterface
+}
+
+func GetElasticsearchIndexerInterface() ElasticsearchIndexerInterface {
+	return theElasticsearchIndexerInterface
+}

vendor/github.com/mattermost/platform/einterfaces/ldap.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2015 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package einterfaces

vendor/github.com/mattermost/platform/einterfaces/metrics.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2015 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package einterfaces
@@ -33,6 +33,7 @@ type MetricsInterface interface {
 	IncrementMemCacheHitCounterSession()
 
 	IncrementWebsocketEvent(eventType string)
+	IncrementWebSocketBroadcast(eventType string)
 
 	AddMemCacheHitCounter(cacheName string, amount float64)
 	AddMemCacheMissCounter(cacheName string, amount float64)

vendor/github.com/mattermost/platform/einterfaces/mfa.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2016 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2016-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package einterfaces

vendor/github.com/mattermost/platform/einterfaces/oauthproviders.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2015 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package einterfaces

vendor/github.com/mattermost/platform/einterfaces/saml.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2015 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package einterfaces
@@ -10,7 +10,7 @@ import (
 type SamlInterface interface {
 	ConfigureSP() *model.AppError
 	BuildRequest(relayState string) (*model.SamlAuthRequest, *model.AppError)
-	DoLogin(encodedXML string, relayState map[string]string, siteURL string) (*model.User, *model.AppError)
+	DoLogin(encodedXML string, relayState map[string]string) (*model.User, *model.AppError)
 	GetMetadata() (string, *model.AppError)
 }
 

vendor/github.com/mattermost/platform/model/LICENSE.txt

@@ -1,32 +1,32 @@
 Mattermost Licensing
 
-SOFTWARE LICENSING 
+SOFTWARE LICENSING
 
-You are licensed to use compiled versions of the Mattermost platform produced by Mattermost, Inc. under an MIT LICENSE 
+You are licensed to use compiled versions of the Mattermost platform produced by Mattermost, Inc. under an MIT LICENSE
 
 -	See MIT-COMPILED-LICENSE.md included in compiled versions for details
 
 You may be licensed to use source code to create compiled versions not produced by Mattermost, Inc. in one of two ways:
 
-1. Under the Free Software Foundation’s GNU AGPL v.3.0, subject to the exceptions outlined in this policy; or 
-2. Under a commercial license available from Mattermost, Inc. by contacting commercial@mattermost.com 
+1. Under the Free Software Foundation’s GNU AGPL v.3.0, subject to the exceptions outlined in this policy; or
+2. Under a commercial license available from Mattermost, Inc. by contacting commercial@mattermost.com
 
-You are licensed to use the source code in Admin Tools and Configuration Files (templates/, config/, model/, 
+You are licensed to use the source code in Admin Tools and Configuration Files (templates/, config/, model/,
 webapp/client, webapp/fonts, webapp/i18n, webapp/images and all subdirectories thereof) under the Apache License v2.0.
 
-We promise that we will not enforce the copyleft provisions in AGPL v3.0 against you if your application (a) does not 
+We promise that we will not enforce the copyleft provisions in AGPL v3.0 against you if your application (a) does not
 link to the Mattermost Platform directly, but exclusively uses the Mattermost Admin Tools and Configuration Files, and
-(b) you have not modified, added to or adapted the source code of Mattermost in a way that results in the creation of 
+(b) you have not modified, added to or adapted the source code of Mattermost in a way that results in the creation of
 a “modified version” or “work based on” Mattermost as these terms are defined in the AGPL v3.0 license.
 
 MATTERMOST TRADEMARK GUIDELINES
 
-Your use of the mark Mattermost is subject to Mattermost, Inc's prior written approval and our organization’s Trademark 
-Standards of Use at http://www.mattermost.org/trademark-standards-of-use/. For trademark approval or any questions 
-you have about using these trademarks, please email trademark@mattermost.com 
+Your use of the mark Mattermost is subject to Mattermost, Inc's prior written approval and our organization’s Trademark
+Standards of Use at http://www.mattermost.org/trademark-standards-of-use/. For trademark approval or any questions
+you have about using these trademarks, please email trademark@mattermost.com
 
 ------------------------------------------------------------------------------------------------------------------------------
-                               
+
                                Apache License
                            Version 2.0, January 2004
                         http://www.apache.org/licenses/

vendor/github.com/mattermost/platform/model/access.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2015 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package model
@@ -21,6 +21,7 @@ type AccessData struct {
 	RefreshToken string `json:"refresh_token"`
 	RedirectUri  string `json:"redirect_uri"`
 	ExpiresAt    int64  `json:"expires_at"`
+	Scope        string `json:"scope"`
 }
 
 type AccessResponse struct {
@@ -51,7 +52,7 @@ func (ad *AccessData) IsValid() *AppError {
 		return NewLocAppError("AccessData.IsValid", "model.access.is_valid.refresh_token.app_error", nil, "")
 	}
 
-	if len(ad.RedirectUri) > 256 {
+	if len(ad.RedirectUri) == 0 || len(ad.RedirectUri) > 256 || !IsValidHttpUrl(ad.RedirectUri) {
 		return NewLocAppError("AccessData.IsValid", "model.access.is_valid.redirect_uri.app_error", nil, "")
 	}
 

vendor/github.com/mattermost/platform/model/analytics_row.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2015 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package model

vendor/github.com/mattermost/platform/model/audit.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2015 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package model

vendor/github.com/mattermost/platform/model/audits.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2015 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package model

vendor/github.com/mattermost/platform/model/authorization.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2016 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2016-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package model
@@ -39,6 +39,7 @@ var PERMISSION_DELETE_PUBLIC_CHANNEL *Permission
 var PERMISSION_DELETE_PRIVATE_CHANNEL *Permission
 var PERMISSION_EDIT_OTHER_USERS *Permission
 var PERMISSION_READ_CHANNEL *Permission
+var PERMISSION_READ_PUBLIC_CHANNEL *Permission
 var PERMISSION_PERMANENT_DELETE_USER *Permission
 var PERMISSION_UPLOAD_FILE *Permission
 var PERMISSION_GET_PUBLIC_LINK *Permission
@@ -47,6 +48,7 @@ var PERMISSION_MANAGE_OTHERS_WEBHOOKS *Permission
 var PERMISSION_MANAGE_OAUTH *Permission
 var PERMISSION_MANAGE_SYSTEM_WIDE_OAUTH *Permission
 var PERMISSION_CREATE_POST *Permission
+var PERMISSION_CREATE_POST_PUBLIC *Permission
 var PERMISSION_EDIT_POST *Permission
 var PERMISSION_EDIT_OTHERS_POSTS *Permission
 var PERMISSION_DELETE_POST *Permission
@@ -56,6 +58,11 @@ var PERMISSION_CREATE_TEAM *Permission
 var PERMISSION_MANAGE_TEAM *Permission
 var PERMISSION_IMPORT_TEAM *Permission
 var PERMISSION_VIEW_TEAM *Permission
+var PERMISSION_LIST_USERS_WITHOUT_TEAM *Permission
+var PERMISSION_MANAGE_JOBS *Permission
+var PERMISSION_CREATE_USER_ACCESS_TOKEN *Permission
+var PERMISSION_READ_USER_ACCESS_TOKEN *Permission
+var PERMISSION_REVOKE_USER_ACCESS_TOKEN *Permission
 
 // General permission that encompases all system admin functions
 // in the future this could be broken up to allow access to some
@@ -64,9 +71,14 @@ var PERMISSION_MANAGE_SYSTEM *Permission
 
 var ROLE_SYSTEM_USER *Role
 var ROLE_SYSTEM_ADMIN *Role
+var ROLE_SYSTEM_POST_ALL *Role
+var ROLE_SYSTEM_POST_ALL_PUBLIC *Role
+var ROLE_SYSTEM_USER_ACCESS_TOKEN *Role
 
 var ROLE_TEAM_USER *Role
 var ROLE_TEAM_ADMIN *Role
+var ROLE_TEAM_POST_ALL *Role
+var ROLE_TEAM_POST_ALL_PUBLIC *Role
 
 var ROLE_CHANNEL_USER *Role
 var ROLE_CHANNEL_ADMIN *Role
@@ -195,6 +207,11 @@ func InitalizePermissions() {
 		"authentication.permissions.read_channel.name",
 		"authentication.permissions.read_channel.description",
 	}
+	PERMISSION_READ_PUBLIC_CHANNEL = &Permission{
+		"read_public_channel",
+		"authentication.permissions.read_public_channel.name",
+		"authentication.permissions.read_public_channel.description",
+	}
 	PERMISSION_PERMANENT_DELETE_USER = &Permission{
 		"permanent_delete_user",
 		"authentication.permissions.permanent_delete_user.name",
@@ -235,6 +252,11 @@ func InitalizePermissions() {
 		"authentication.permissions.create_post.name",
 		"authentication.permissions.create_post.description",
 	}
+	PERMISSION_CREATE_POST_PUBLIC = &Permission{
+		"create_post_public",
+		"authentication.permissions.create_post_public.name",
+		"authentication.permissions.create_post_public.description",
+	}
 	PERMISSION_EDIT_POST = &Permission{
 		"edit_post",
 		"authentication.permissions.edit_post.name",
@@ -280,6 +302,31 @@ func InitalizePermissions() {
 		"authentication.permissions.view_team.name",
 		"authentication.permissions.view_team.description",
 	}
+	PERMISSION_LIST_USERS_WITHOUT_TEAM = &Permission{
+		"list_users_without_team",
+		"authentication.permissions.list_users_without_team.name",
+		"authentication.permissions.list_users_without_team.description",
+	}
+	PERMISSION_CREATE_USER_ACCESS_TOKEN = &Permission{
+		"create_user_access_token",
+		"authentication.permissions.create_user_access_token.name",
+		"authentication.permissions.create_user_access_token.description",
+	}
+	PERMISSION_READ_USER_ACCESS_TOKEN = &Permission{
+		"read_user_access_token",
+		"authentication.permissions.read_user_access_token.name",
+		"authentication.permissions.read_user_access_token.description",
+	}
+	PERMISSION_REVOKE_USER_ACCESS_TOKEN = &Permission{
+		"revoke_user_access_token",
+		"authentication.permissions.revoke_user_access_token.name",
+		"authentication.permissions.revoke_user_access_token.description",
+	}
+	PERMISSION_MANAGE_JOBS = &Permission{
+		"manage_jobs",
+		"authentication.permisssions.manage_jobs.name",
+		"authentication.permisssions.manage_jobs.description",
+	}
 }
 
 func InitalizeRoles() {
@@ -293,7 +340,6 @@ func InitalizeRoles() {
 		[]string{
 			PERMISSION_READ_CHANNEL.Id,
 			PERMISSION_MANAGE_PUBLIC_CHANNEL_MEMBERS.Id,
-			PERMISSION_MANAGE_PRIVATE_CHANNEL_MEMBERS.Id,
 			PERMISSION_UPLOAD_FILE.Id,
 			PERMISSION_GET_PUBLIC_LINK.Id,
 			PERMISSION_CREATE_POST.Id,
@@ -326,17 +372,38 @@ func InitalizeRoles() {
 		[]string{
 			PERMISSION_LIST_TEAM_CHANNELS.Id,
 			PERMISSION_JOIN_PUBLIC_CHANNELS.Id,
+			PERMISSION_READ_PUBLIC_CHANNEL.Id,
 			PERMISSION_VIEW_TEAM.Id,
 		},
 	}
 	BuiltInRoles[ROLE_TEAM_USER.Id] = ROLE_TEAM_USER
+
+	ROLE_TEAM_POST_ALL = &Role{
+		"team_post_all",
+		"authentication.roles.team_post_all.name",
+		"authentication.roles.team_post_all.description",
+		[]string{
+			PERMISSION_CREATE_POST.Id,
+		},
+	}
+	BuiltInRoles[ROLE_TEAM_POST_ALL.Id] = ROLE_TEAM_POST_ALL
+
+	ROLE_TEAM_POST_ALL_PUBLIC = &Role{
+		"team_post_all_public",
+		"authentication.roles.team_post_all_public.name",
+		"authentication.roles.team_post_all_public.description",
+		[]string{
+			PERMISSION_CREATE_POST_PUBLIC.Id,
+		},
+	}
+	BuiltInRoles[ROLE_TEAM_POST_ALL_PUBLIC.Id] = ROLE_TEAM_POST_ALL_PUBLIC
+
 	ROLE_TEAM_ADMIN = &Role{
 		"team_admin",
 		"authentication.roles.team_admin.name",
 		"authentication.roles.team_admin.description",
 		[]string{
 			PERMISSION_EDIT_OTHERS_POSTS.Id,
-			PERMISSION_ADD_USER_TO_TEAM.Id,
 			PERMISSION_REMOVE_USER_FROM_TEAM.Id,
 			PERMISSION_MANAGE_TEAM.Id,
 			PERMISSION_IMPORT_TEAM.Id,
@@ -358,10 +425,42 @@ func InitalizeRoles() {
 			PERMISSION_CREATE_DIRECT_CHANNEL.Id,
 			PERMISSION_CREATE_GROUP_CHANNEL.Id,
 			PERMISSION_PERMANENT_DELETE_USER.Id,
-			PERMISSION_MANAGE_OAUTH.Id,
 		},
 	}
 	BuiltInRoles[ROLE_SYSTEM_USER.Id] = ROLE_SYSTEM_USER
+
+	ROLE_SYSTEM_POST_ALL = &Role{
+		"system_post_all",
+		"authentication.roles.system_post_all.name",
+		"authentication.roles.system_post_all.description",
+		[]string{
+			PERMISSION_CREATE_POST.Id,
+		},
+	}
+	BuiltInRoles[ROLE_SYSTEM_POST_ALL.Id] = ROLE_SYSTEM_POST_ALL
+
+	ROLE_SYSTEM_POST_ALL_PUBLIC = &Role{
+		"system_post_all_public",
+		"authentication.roles.system_post_all_public.name",
+		"authentication.roles.system_post_all_public.description",
+		[]string{
+			PERMISSION_CREATE_POST_PUBLIC.Id,
+		},
+	}
+	BuiltInRoles[ROLE_SYSTEM_POST_ALL_PUBLIC.Id] = ROLE_SYSTEM_POST_ALL_PUBLIC
+
+	ROLE_SYSTEM_USER_ACCESS_TOKEN = &Role{
+		"system_user_access_token",
+		"authentication.roles.system_user_access_token.name",
+		"authentication.roles.system_user_access_token.description",
+		[]string{
+			PERMISSION_CREATE_USER_ACCESS_TOKEN.Id,
+			PERMISSION_READ_USER_ACCESS_TOKEN.Id,
+			PERMISSION_REVOKE_USER_ACCESS_TOKEN.Id,
+		},
+	}
+	BuiltInRoles[ROLE_SYSTEM_USER_ACCESS_TOKEN.Id] = ROLE_SYSTEM_USER_ACCESS_TOKEN
+
 	ROLE_SYSTEM_ADMIN = &Role{
 		"system_admin",
 		"authentication.roles.global_admin.name",
@@ -378,6 +477,8 @@ func InitalizeRoles() {
 							PERMISSION_MANAGE_SYSTEM.Id,
 							PERMISSION_MANAGE_ROLES.Id,
 							PERMISSION_MANAGE_PUBLIC_CHANNEL_PROPERTIES.Id,
+							PERMISSION_MANAGE_PUBLIC_CHANNEL_MEMBERS.Id,
+							PERMISSION_MANAGE_PRIVATE_CHANNEL_MEMBERS.Id,
 							PERMISSION_DELETE_PUBLIC_CHANNEL.Id,
 							PERMISSION_CREATE_PUBLIC_CHANNEL.Id,
 							PERMISSION_MANAGE_PRIVATE_CHANNEL_PROPERTIES.Id,
@@ -391,6 +492,13 @@ func InitalizeRoles() {
 							PERMISSION_DELETE_POST.Id,
 							PERMISSION_DELETE_OTHERS_POSTS.Id,
 							PERMISSION_CREATE_TEAM.Id,
+							PERMISSION_ADD_USER_TO_TEAM.Id,
+							PERMISSION_LIST_USERS_WITHOUT_TEAM.Id,
+							PERMISSION_MANAGE_JOBS.Id,
+							PERMISSION_CREATE_POST_PUBLIC.Id,
+							PERMISSION_CREATE_USER_ACCESS_TOKEN.Id,
+							PERMISSION_READ_USER_ACCESS_TOKEN.Id,
+							PERMISSION_REVOKE_USER_ACCESS_TOKEN.Id,
 						},
 						ROLE_TEAM_USER.Permissions...,
 					),

vendor/github.com/mattermost/platform/model/authorize.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2015 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package model
@@ -6,6 +6,7 @@ package model
 import (
 	"encoding/json"
 	"io"
+	"net/http"
 )
 
 const (
@@ -25,6 +26,14 @@ type AuthData struct {
 	Scope       string `json:"scope"`
 }
 
+type AuthorizeRequest struct {
+	ResponseType string `json:"response_type"`
+	ClientId     string `json:"client_id"`
+	RedirectUri  string `json:"redirect_uri"`
+	Scope        string `json:"scope"`
+	State        string `json:"state"`
+}
+
 // IsValid validates the AuthData and returns an error if it isn't configured
 // correctly.
 func (ad *AuthData) IsValid() *AppError {
@@ -49,7 +58,7 @@ func (ad *AuthData) IsValid() *AppError {
 		return NewLocAppError("AuthData.IsValid", "model.authorize.is_valid.create_at.app_error", nil, "client_id="+ad.ClientId)
 	}
 
-	if len(ad.RedirectUri) > 256 {
+	if len(ad.RedirectUri) == 0 || len(ad.RedirectUri) > 256 || !IsValidHttpUrl(ad.RedirectUri) {
 		return NewLocAppError("AuthData.IsValid", "model.authorize.is_valid.redirect_uri.app_error", nil, "client_id="+ad.ClientId)
 	}
 
@@ -64,6 +73,33 @@ func (ad *AuthData) IsValid() *AppError {
 	return nil
 }
 
+// IsValid validates the AuthorizeRequest and returns an error if it isn't configured
+// correctly.
+func (ar *AuthorizeRequest) IsValid() *AppError {
+
+	if len(ar.ClientId) != 26 {
+		return NewAppError("AuthData.IsValid", "model.authorize.is_valid.client_id.app_error", nil, "", http.StatusBadRequest)
+	}
+
+	if len(ar.ResponseType) == 0 {
+		return NewAppError("AuthData.IsValid", "model.authorize.is_valid.response_type.app_error", nil, "", http.StatusBadRequest)
+	}
+
+	if len(ar.RedirectUri) == 0 || len(ar.RedirectUri) > 256 || !IsValidHttpUrl(ar.RedirectUri) {
+		return NewAppError("AuthData.IsValid", "model.authorize.is_valid.redirect_uri.app_error", nil, "client_id="+ar.ClientId, http.StatusBadRequest)
+	}
+
+	if len(ar.State) > 128 {
+		return NewAppError("AuthData.IsValid", "model.authorize.is_valid.state.app_error", nil, "client_id="+ar.ClientId, http.StatusBadRequest)
+	}
+
+	if len(ar.Scope) > 128 {
+		return NewAppError("AuthData.IsValid", "model.authorize.is_valid.scope.app_error", nil, "client_id="+ar.ClientId, http.StatusBadRequest)
+	}
+
+	return nil
+}
+
 func (ad *AuthData) PreSave() {
 	if ad.ExpiresIn == 0 {
 		ad.ExpiresIn = AUTHCODE_EXPIRE_TIME
@@ -98,6 +134,26 @@ func AuthDataFromJson(data io.Reader) *AuthData {
 	}
 }
 
+func (ar *AuthorizeRequest) ToJson() string {
+	b, err := json.Marshal(ar)
+	if err != nil {
+		return ""
+	} else {
+		return string(b)
+	}
+}
+
+func AuthorizeRequestFromJson(data io.Reader) *AuthorizeRequest {
+	decoder := json.NewDecoder(data)
+	var ar AuthorizeRequest
+	err := decoder.Decode(&ar)
+	if err == nil {
+		return &ar
+	} else {
+		return nil
+	}
+}
+
 func (ad *AuthData) IsExpired() bool {
 
 	if GetMillis() > ad.CreateAt+int64(ad.ExpiresIn*1000) {

vendor/github.com/mattermost/platform/model/channel.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2015 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package model
@@ -46,6 +46,13 @@ type Channel struct {
 	CreatorId     string `json:"creator_id"`
 }
 
+type ChannelPatch struct {
+	DisplayName *string `json:"display_name"`
+	Name        *string `json:"name"`
+	Header      *string `json:"header"`
+	Purpose     *string `json:"purpose"`
+}
+
 func (o *Channel) ToJson() string {
 	b, err := json.Marshal(o)
 	if err != nil {
@@ -55,6 +62,15 @@ func (o *Channel) ToJson() string {
 	}
 }
 
+func (o *ChannelPatch) ToJson() string {
+	b, err := json.Marshal(o)
+	if err != nil {
+		return ""
+	} else {
+		return string(b)
+	}
+}
+
 func ChannelFromJson(data io.Reader) *Channel {
 	decoder := json.NewDecoder(data)
 	var o Channel
@@ -66,6 +82,17 @@ func ChannelFromJson(data io.Reader) *Channel {
 	}
 }
 
+func ChannelPatchFromJson(data io.Reader) *ChannelPatch {
+	decoder := json.NewDecoder(data)
+	var o ChannelPatch
+	err := decoder.Decode(&o)
+	if err == nil {
+		return &o
+	} else {
+		return nil
+	}
+}
+
 func (o *Channel) Etag() string {
 	return Etag(o.Id, o.UpdateAt)
 }
@@ -137,6 +164,24 @@ func (o *Channel) IsGroupOrDirect() bool {
 	return o.Type == CHANNEL_DIRECT || o.Type == CHANNEL_GROUP
 }
 
+func (o *Channel) Patch(patch *ChannelPatch) {
+	if patch.DisplayName != nil {
+		o.DisplayName = *patch.DisplayName
+	}
+
+	if patch.Name != nil {
+		o.Name = *patch.Name
+	}
+
+	if patch.Header != nil {
+		o.Header = *patch.Header
+	}
+
+	if patch.Purpose != nil {
+		o.Purpose = *patch.Purpose
+	}
+}
+
 func GetDMNameFromIds(userId1, userId2 string) string {
 	if userId1 > userId2 {
 		return userId2 + "__" + userId1

vendor/github.com/mattermost/platform/model/channel_count.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2015 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package model

vendor/github.com/mattermost/platform/model/channel_data.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2015 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package model

vendor/github.com/mattermost/platform/model/channel_list.go

@@ -1,4 +1,4 @@
-// Copyright (c) 2015 Mattermost, Inc. All Rights Reserved.
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
 package model
@@ -50,3 +50,14 @@ func ChannelListFromJson(data io.Reader) *ChannelList {
 		return nil
 	}
 }
+
+func ChannelSliceFromJson(data io.Reader) []*Channel {
+	decoder := json.NewDecoder(data)
+	var o []*Channel
+	err := decoder.Decode(&o)
+	if err == nil {
+		return o
+	} else {
+		return nil
+	}
+}