-
Notifications
You must be signed in to change notification settings - Fork 5
Implement websocket client #13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
I am still somewhat new to the codebase, so this is a general question and not feedback on your PR (which looks solid btw), but how does this prevent the code from sending messages over the standard socket connection? Judging by the naming and the hookCall source code it seems that hookCall doesn't replace the method, just allows executing additional code before/after it. |
|
Also, hostname might not be the clearest name for the path of the WebSocket connection, since it could be a route on a server (for example, |
The The first argument in the handler is ignored in my implementation. I simply don't call the original method. However, working on tests I've discovered this approach has one key issue. The original client provides a
Good catch, I'll make the adjustment. |
|
Awesome PR 👍 I would not have thought of doing this with a mix of plugin/hooks but I find this way to go very interesting. I initially thought of a common "transport layer" used by the core client which could be seen as an "adapter" in order to abstract TCP and WebSockets implementations. For instance, the existing code related to TCP would be moved to But now I wonder if it would not be more relevant/easier of doing it with plugin. Maybe the WebSocket support should be seen as an extension and not a core feature. 🤔
Currently it works like following:
|
|
Just an update with some thoughts. Working on this again this evening, plan to push some tests, not sure it'll be complete but we'll see. Still need to figure out how to handle the websocket close on disconnect or else the library will leak promises when torn down. I do think websocket support is best as a pluggable feature given the nature of it being pluggable on the remote side and in the spec in general. It is in an interesting place where it is a transport and not a protocol level feature as mentioned before, but I think architecting it this way makes the most sense (the I'm going to remove the use of the WebSocketClient/Server library I imported. A colleague showed me how to do it with just Deno standard libraries and the third-party library is proving difficult to mock. I also plan to bring in npm:mock-socket for the testing effort. |
|
It appears removing the 3rd party library has solved the promise issue. |
|
I think we're ready for review. I added a test scenario covering initial connection and registration with a mock websocket server. Also snuck in a fix to lower the wait times on the antiflood test to make it execute in a quarter of a second instead of one. |
|
A few questions:
|
|
Thanks both of you for the information on hooks by the way, very helpful!
Makes sense, the WebSocket API is event-based and not async-await (just like our library actually). I did wonder why you pulled in the third-party dep but figured the reasoning was just that websockets are just awkward to use (full duplex and all that).
Agreed, and this particular one is quite a clever implementation as well. |
I saw it in some example configurations for Ergo when researching the protocol. It comes from the typical convention of prepending the HTTP port
It's only used within the original codebase inside the original TCP client, its test, and for printing the "server hostname" (in this case now "server URL"). This state is exposed on the public API surface so renaming it may be undesirable. We could remove its use entirely in the websocket client, or we could indeed parse out the hostname and expose a new property
I found with the existing server implementation I tested with we didn't have to -- Ergo at least assumes the protocol from our first payload and we're sending it in the MUST implement binary format. The spec guides servers to support clients that don't implement the negotiation. I'll be able to make these changes later today. |
|
Seems I was mistaken, I did not implement the protocol correctly. The binary protocol is now implemented as well, and negotiated first. I tested against all three server implementations through discord-irc confirming messages are passed and parsed.
I found this excellent IRC websocket tester site that includes minimal code examples as well as test URLs to register with implementations of these IRC servers using websockets. The minimal example fails as well... function delay(ms = 0) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
let socket = new WebSocket('wss://testnet.inspircd.org:8097', ['binary.ircv3.net', 'text.ircv3.net']);
socket.addEventListener('error', error => {
console.log(error);
});
socket.addEventListener('open', () => {
try {
socket.send('NICK Example9999');
socket.send('USER 0 0 0 0');
} catch (e) {
console.log(e);
}
});
socket.addEventListener('message', async e => {
var line = e.data;
if (line instanceof Blob) {
line = await e.data.text();
}
console.log(line);
})
await delay(10000);Not sure what to make of this. Perhaps we open an issue upstream? |
|
It looks like an issue with the WebSocket implementation in Deno? |
|
I'll have to wireshark the HTTP interaction during the websocket upgrade, but I'm inclined to think this is a Deno correctness feature where invalid upgrade attempts are marked as invalid, whereas Node would just allow them anyways. This might be an InspIRCd bug if the upgrade exchange proves to indeed send the wrong status code. Don't have enough evidence yet to know for sure. |
|
Thank you for taking a look 🙏 The fact that the same example with Node.js fails tends to confirm that the cause comes from the ircd. 👍 |
|
I'm OK with merging this now after your review. I've addressed some issues in the original implementation and we have solid tests. I added some unit tests for the text-protocol-only case as well as testing some of the error handlers and internal state cleanup code. I got it to 81.818% (81/99 lines) coverage. All that's left are catch blocks I'm having trouble getting to trip. |
|
One last-minute update before review. I had an oversight in testing websocket connections that have a path. I've added an optional parameter to |
|
InspIRCd issue is tracked here |
|
Update after talking with the maintainer of InspIRCd: This "issue" in InspIRCd is not really an issue per se, it's a logical misuse of the websocket feature. Websocket support is intended for browser environments. We're (Deno, really) not sending an So, if we ran this library in a web browser, the connection to InspIRCd would work (or be rejected by the server configuration) as the |
Some parts are hard to test and don't add much value. Nominal case is the most important for now 👍 |
I read the issue so the good place for WebSocket support is indeed in a plugin. Although the final purpose of it is to work in a browser (not in Node.js or Deno), it's great to have it for our lib. |
|
I have a little API suggestion about async connect(
hostname: string,
port = PORT,
tls = false,
_path?: string,
): Promise<Deno.Conn | null>
client.connect("irc.example.org", 8097, true, "pathTo/Irc");I should have change it before, but now that we add extra argument, I wonder if all optional arguments could not be passed in an object option: type ConnectOptions = {
port?: number
tls?: boolean;
path?: string;
}
async connect(
hostname: string,
options: ConnectOptions = {}
): Promise<Deno.Conn | null> {
options.port ??= PORT
options.tls ??= false
...
}
// usages
client.connect("irc.example.org", { port: 8097, tls: true });
client.connect("irc.example.org", { path: "pathTo/Irc" });WDYT? Not sure about including optional port since we often change it (especially when SSL is enabled). |
|
I think that's a solid change. If we're going to change the public API surface we should use this Reading that section of your link, I agree with the guideline.
I think it's best for the port to be required. The port's default state would otherwise also rely on the potentially undefined state of |
I think you are right. Due to the multiple choices we have now (TCP, secure TCP, WebSockets, ...), maybe having the port mandatory would be a good thing in order to make it more explicit. Otherwise I initially thought of having these usages with some predefined ports: const client = new Client({ ... }); // websocket: false
client.connect("irc.example.org"); // use default port 6667
client.connect("irc.example.org", { tls: true }); // use default port 6697
client.connect("irc.example.org", { tls: true, port: 7000 }); // use custom port
const client = new Client({ websocket: true });
client.connect("irc.example.org", { path: "pathTo/Irc" }); // use default port 80
client.connect("irc.example.org", { path: "pathTo/Irc", tls: true }); // use default port 443
client.connect("irc.example.org", { path: "pathTo/Irc", tls: true, port: 8097 }); // use custom port But, especially for WebSocket cases, making the port optional could probably cause unexpected errors. |
|
Sorry for the delay in response and my lack of involvement with this feature in general, been kinda slammed this week with life stuff.
Honestly it sounds like a bug for even farther upstream. If Deno promises browser environment compat Re: the WebSocket port number, I'd love if we could infer it, but the jankiness with 80/443/8067/8097/ something else entirely sways me toward just making it a required param. Maybe we could explain it in the docs or print a nice error with some values for ports (although I think anyone mucking about this deep on the IRC iceberg already has an idea what port number to use).
Offtopic, but how do you measure codecov? Do you just calculate it manually? |
|
Okay, now I feel stupid. |
On second thought, from an ergonomic standpoint, I don't know if it'd be the best thing. We would create an API difference between WebSockets and TCP transports which is not strictly necessary (when I say strictly necessary, I mean the To be fair this is a pretty contrived usecase, but if I were working on TCP and then realised I needed to be using WebSockets to connect, I'd expect to add But then again, there's no sane default port number for IRC-over-WebSockets. Is this a bug in the spec? :3 |
|
Don't worry about any delay, we're all volunteers just trying to make this better. I don't think there's any rush!
This may be true. I was somewhat astonished we couldn't set the Origin ourselves without re-implementing websockets from the ground up. I will draft up something to put on their tracker. We'll see what upstream thinks.
My thoughts exactly. Either they've set up the server themselves, or they're likely already active on the server they want to connect to and therefore have already figured out the details. I don't think anyone is using this library in a vacuum. When I'm free later today I'll make the port required as the second parameter and implement
I don't think this is a reasonable expectation. Users may carry this expectation, but it's not aligned with the reality that there is no solid, standard port for websocket connections for IRC. There is the HTTP(s) standard and the 8000+(67/97) standard. Which standard do they expect would work? Whichever we decide on, we're technically wrong, or may mislead a user. I think it's better to be agnostic of port and make it required. This also helps guide the user towards the Keep in mind, as the InspIRCd maintainer mentioned in the issue I linked,
IRC network admins should know exactly which port they need. |
I'm gonna ask in the ircv3 channel their thoughts on adding a "suggested" port number to the spec. The core IRC spec mentions the standard TCP port number, so why shouldn't the WebSocket extension? |
I could handle that, if you'd like. |
|
Sorry for the comment spam, but I have more thoughts to add and I'm not sure what the better etiquette is here, add a new comment or edit an existing one. But anyway, I see why Deno maintainers may have done this - Origin (along with many other headers) is marked as forbidden, and the browser WebSocket API doesn't allow setting it. So I see why they don't let us set it, but at the same time, it breaks web applications that expect an Origin header in the WS upgrade request. They're kind of stuck between a rock and a hard place. |
Also, do we want to bother with this portion of the spec? This is from the non-normative section. |
K, so as far as the spec goes, it's pretty much a WONTFIX as far as I can tell, which is fair on their part. However, a lot of people do seem to agree that 80/443 are the "expected" behaviour. I say we do what emersion says and let the WS implementation in deno handle the port number based on what protocol we pass, and only include the port in the connection URL if it's supplied. I guess that presents an issue for the remoteAddr side of things though. |
Apparently there's a Relevant docs: WebSocketStreamOptions and WebSocketStream. It seems pretty barebones though. |
|
I think they're approaching this from a different angle than we are. Typically one provides a URL for a websocket connection. The URL would imply or provide the port. However, in our case, we are not passing/passed the URL. We pass/are passed the components of the URL in the public API surface, which means it must be assembled. Therefore none of the guarantees built into the websocket URL format exist here. There is no "expected behavior" other than what's implied by the API surface and documented IMHO. And we can comment the API surface to make it obvious in developers' IDEs. It's not going to be obvious to everyone that the hostname and path are assembled into a websocket URL internally -- which is where the default port would come from -- nor should they have to care IMO. While I now think port 80/443 are acceptable defaults, I don't think it should be optional. I think I'd want a developer who is trying to make a connection to IRC in their first project with this library to get an error when they make a call like The alternate situation is a developer who read the documentation, where we can have a section explaining which ports are typical 🙂 I like the examples Jerome wrote for that.
I don't think I do, unless we can find such a legacy server to test against. The three existing implementations are what will be used by a majority of servers I'm sure. Other servers aren't developed yet or are still in development AFAIK, so when they do get developed they'll be aware of subprotocols. If we did implement this, we'd have to detect if it's binary or text protocol and I'm not sure the best way to do that. If you want to add a commit implementing this go for it!
👀 have all our problems been solved? It's unstable though, I don't think we should include it just yet. Perhaps a future enhancement. |


Hello, I bring an update discussed in issue #12
It connects to IRC servers through the IRCv3 WebSocket specification.
This PR is a WIP and is not in a ready state. It still needs unit tests and documentation.
Tested working:
This PR has been tested with a local Ergo IRCv3 server and discord-irc running locally and all features work as expected.