First, from the aspect of grammaticality or idiomaticity:
首先，從語法性或慣用性的角度
"My ... is poor" isn't very idiomatic in English.
「My ... is poor」在英語中並不是很地道
"My swimming is poor" well... "I'm a poor swimmer" or "I'm not a very good swimmer" might be better.
你說「My swimming is poor」不如說「I'm a poor swimmer」或者「I'm not a very good swimmer」。
"My speaking skills are poor"... How about "I'm a poor speaker" or "I'm not good at speaking"?
「My speaking skills are poor」...不如說成「I'm a poor speaker」或者「I'm not good at speaking」。
...
In each case, using "poor" as a predicative adjective is not very idiomatic. The more idiomatic alternatives are to say "I'm poor at...", "I'm not good at...", "My ... isn't very good".
在每種情況下，用poor作表語來形容都不是慣用的用法。更地道的說法是「I'm poor at...」,「I'm not good at...」，「My ... isn't very good」。
If you want to include slangy expressions, you might say "My English sucks" --- but if your English was good enough to say that, you wouldn't need to say it at all!
如果你想用一些俚語表達，你可能會說「My English sucks」——但是得是你英語很好才會用到這樣俚語表達，側面也就說明了你根本就不用說你英語不好。
Second, from the cultural or sociolinguistic aspect:
第二，從文化或社會語言學的角度
Telling people your English is poor is just a statement; saying it won't necessarily get the effect that you want.
告訴別人你的英語很差只是一種聲明；說它不一定能達到你想要的效果。
If you say "My English is poor" ，you might only succeed in getting people to say "No, your English is fine! Better than mine!", but that's not what you want.
如果你說「My English is poor」，那麼你很有可能會成功得到別人回覆你說「不，你英語挺好的！比我好！」，但是這並不是你真正想要的效果。
If you want the other person to be aware that you're still struggling with English and to make allowances for that, there are probably better ways of putting it.
如果你想讓對方知道你的英語還很差，並體諒一下，可能還有更好的表達方式。
For instance, "Sorry, I haven't been learning English for very long, could you (...say that again please, ... slow down a bit, etc.)" or even "I'm still just learning English".
例如「對不起，我很久沒學英語了，你能...（再說一遍...說慢點等等）嗎？」或者「我還在學英語」。
If you want to signal that your expression may be awkward or you're not sure if you will be taken the right way, you might say "Sorry, I'm not sure how to say this in English but..."
如果你想表示你的說法可能很尷尬，或者你不確定你是否被正確理解，你可以這樣說「對不起，我並不確定用英語如何說...」
And in fact, if your English is really that "poor", people will pick up on it quite quickly. It may not be necessary to say anything at all.
事實上，如果你的英語真的那麼差，人們很快就會反映過來。也許根本沒有必要說什麼。